LFCE 60 🐧 Load Balancing — Concepts - KandZ

⏱️ Estimated read: 13 min | |

Load balancing is the distribution of incoming network traffic across multiple backend servers. The purpose is not to make a single request faster; it is to prevent any one server from becoming a bottleneck, to provide redundancy when a server fails, and to allow the pool to scale by adding more servers rather than upgrading one. On Linux, load balancing appears at two distinct layers with different tools. At Layer 4, the kernel’s IPVS module distributes TCP and UDP connections based on packet headers. At Layer 7, a userspace proxy like HAProxy or Nginx inspects the HTTP request and routes it based on URL, headers, or cookies . The previous chapters covered the reverse proxies individually; this chapter covers the concepts that are common to all load-balancing implementations: the algorithms, the routing methods, the health checks, and the high-availability layer that prevents the load balancer itself from becoming a single point of failure.

Key point: A load balancer must itself be highly available, or it becomes the single point of failure it was meant to eliminate. The standard Linux pattern pairs the load balancer with Keepalived, which uses VRRP to move a virtual IP address between a primary and a backup node. The virtual IP is the address that clients connect to, and it moves to the backup when the primary fails . Without this layer, a single load balancer is a single point of failure regardless of how many backends it fronts.

Why load balancing requires a layered design

The Layer 4 versus Layer 7 problem.

A Layer 4 load balancer forwards packets based on IP address, port, and protocol. It does not inspect the payload, so it can handle any TCP or UDP protocol—HTTP, databases, DNS, mail—without modification. The Linux kernel’s IPVS is the Layer 4 implementation, and it operates in the kernel’s network stack, which gives it very high throughput . A Layer 7 load balancer terminates the connection, parses the application protocol, and makes routing decisions based on content. It can route by URL path, inspect headers, and maintain session affinity through cookies. HAProxy and Nginx are the Layer 7 implementations. The choice between the two layers depends on whether the routing decision requires application-layer information .

The connection-tracking problem.

A Layer 4 load balancer must remember which backend a connection was assigned to, so that subsequent packets in the same connection reach the same backend. IPVS maintains a connection table in the kernel. When a client opens a connection, IPVS selects a backend and records the mapping; subsequent packets in that connection are forwarded to the same backend without re-evaluating the algorithm . This is why Layer 4 load balancing is connection-oriented: the algorithm runs once per connection, not once per packet.

The session-persistence problem.

Some applications store session state locally on the backend. If the load balancer distributes requests from the same client across different backends, the session is lost. The solution is session persistence: the load balancer consistently routes requests from the same client to the same backend. The IP-hash algorithm does this by hashing the client’s IP address . A more reliable approach is to store the session in a shared store like Redis, so that any backend can serve any request and the load balancer is free to distribute requests by any algorithm.

a. Load-balancing algorithms

The algorithm determines how the load balancer selects a backend for each new connection or request. The choice depends on the workload characteristics: the request duration, the backend capacity, and whether session persistence is required.

# IPVS scheduling algorithms
# rr: round-robin
sudo ipvsadm -A -t 192.168.1.100:80 -s rr

# wrr: weighted round-robin
sudo ipvsadm -A -t 192.168.1.100:80 -s wrr

# lc: least-connection
sudo ipvsadm -A -t 192.168.1.100:80 -s lc

# wlc: weighted least-connection (default)
sudo ipvsadm -A -t 192.168.1.100:80 -s wlc

Round-robin assigns each new connection to the next backend in sequence. It works best when the backends have similar capacity and the connections have similar duration. Weighted round-robin gives each backend a share proportional to its weight: a backend with weight 3 receives three times as many connections as a backend with weight 1 . Least-connection assigns the new connection to the backend with the fewest active connections. It is appropriate when connections have variable duration, because it avoids sending new connections to a backend that is already busy. Weighted least-connection combines the two: it divides the active connection count by the weight and selects the backend with the lowest result .

# HAProxy algorithms
# roundrobin: round-robin with weights
# leastconn: least connections
# source: IP hash
# uri: hash of the URI
balance roundrobin
balance leastconn
balance source

HAProxy exposes the same algorithms at Layer 7. The source algorithm hashes the client’s IP address, which provides session persistence for applications that store state locally . The uri algorithm hashes the request URI, which is useful for caching proxies where the same URI should reach the same cache.

The study comparing HAProxy algorithms found that weighted round-robin achieved the highest connections per second in a test with heterogeneous backends, followed by round-robin, least-connection, and source. The recommendation was weighted round-robin for heterogeneous environments and least-connection for sessions with varying durations .

b. Routing methods: NAT, Direct Routing, and Tunneling

IPVS supports three methods for delivering packets to the real servers. The method determines how the response traffic returns to the client and what constraints the topology must satisfy.

# NAT mode: the load balancer rewrites the destination address
sudo ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.10:80 -m

# Direct Routing mode: the load balancer forwards the packet unchanged
sudo ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.10:80 -g

# Tunneling mode: the load balancer encapsulates the packet
sudo ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.10:80 -i

In NAT mode, the load balancer rewrites the destination address of the incoming packet to the backend’s address, and rewrites the source address of the response packet to its own virtual IP. The client sees only the virtual IP. The backend can run any operating system and can use a private address, but all response traffic must pass through the load balancer, which becomes a bottleneck as the cluster grows. The practical limit is around 10 to 20 backend servers .

In Direct Routing mode, the load balancer forwards the packet with the destination address unchanged, and the backend responds directly to the client. The load balancer sees only the incoming traffic, not the outgoing, which removes the bottleneck and allows the cluster to scale beyond 100 servers. The constraint is that the load balancer and the backends must be on the same Layer 2 segment, and the backends must be configured to accept packets for the virtual IP without responding to ARP requests for it . In Tunneling mode, the load balancer encapsulates the packet in an IP tunnel to the backend, which decapsulates it and responds directly to the client. The backends can be on different networks, but they must support the tunneling protocol .

c. Health checks and high availability

A load balancer must detect when a backend has failed and remove it from the pool. It must also be highly available itself, so that its own failure does not take down the service.

# HAProxy health check
backend web_servers
    option httpchk GET /health
    http-check expect status 200
    server web1 192.168.1.11:8080 check
    server web2 192.168.1.12:8080 check

The option httpchk directive tells HAProxy to send an HTTP request to the /health endpoint on each backend. The http-check expect status 200 directive marks the backend as healthy only if the response status is 200. The check keyword on each server enables the health check . A backend that fails the check is removed from rotation; a backend that passes again is returned to the pool.

IPVS does not have a built-in health-check mechanism. The health checks are provided by Keepalived, which runs scripts or TCP checks against the backends and updates the IPVS table when a backend fails .

# Keepalived VRRP instance for a floating IP
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass secret
    }
    virtual_ipaddress {
        192.168.1.100/24
    }
}

The vrrp_instance block defines a virtual router. The state MASTER and priority 100 on the primary node, and state BACKUP and a lower priority on the backup, determine which node holds the virtual IP. The virtual_ipaddress is the address that clients connect to. If the primary fails, the backup takes over the address within the advertisement interval . The virtual_server block, added to the same configuration file, defines the IPVS virtual service and the real servers, and Keepalived programs the kernel’s IPVS table accordingly .

Complete Example Session

# ============================================
# PART 1: INSTALL THE TOOLS
# ============================================
sudo apt install ipvsadm keepalived
# Or: sudo dnf install ipvsadm keepalived

# ============================================
# PART 2: ENABLE IP FORWARDING (NAT MODE)
# ============================================
sudo sysctl -w net.ipv4.ip_forward=1

# ============================================
# PART 3: CREATE AN IPVS VIRTUAL SERVICE
# ============================================
# Round-robin on port 80
sudo ipvsadm -A -t 192.168.1.100:80 -s rr

# ============================================
# PART 4: ADD REAL SERVERS
# ============================================
# NAT mode (-m)
sudo ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.10:80 -m
sudo ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.11:80 -m

# ============================================
# PART 5: VERIFY THE IPVS TABLE
# ============================================
sudo ipvsadm -Ln
# Output shows the virtual service and the real servers

# ============================================
# PART 6: CONFIGURE KEEPALIVED FOR HIGH AVAILABILITY
# ============================================
sudo tee /etc/keepalived/keepalived.conf > /dev/null <<'EOF'
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass secret
    }
    virtual_ipaddress {
        192.168.1.100/24
    }
}

virtual_server 192.168.1.100 80 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT
    protocol TCP

    real_server 192.168.1.10 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
        }
    }
    real_server 192.168.1.11 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
        }
    }
}
EOF

# ============================================
# PART 7: START KEEPALIVED
# ============================================
sudo systemctl enable --now keepalived
sudo systemctl status keepalived

# ============================================
# PART 8: VERIFY THE VIRTUAL IP AND IPVS TABLE
# ============================================
ip addr show eth0 | grep 192.168.1.100
sudo ipvsadm -Ln
# The VIP is on the primary node, and the IPVS table has the real servers

# ============================================
# PART 9: TEST FAILOVER
# ============================================
# Stop keepalived on the primary
sudo systemctl stop keepalived
# On the backup, the VIP moves to its interface
# The backup's IPVS table becomes active

# ============================================
# PART 10: CLEAN UP
# ============================================
sudo systemctl stop keepalived
sudo ipvsadm -C

The ten parts demonstrate the complete Layer 4 load-balancing setup: installation, forwarding, virtual service creation, real server addition, IPVS table verification, Keepalived configuration, service start, VIP verification, failover testing, and cleanup. The critical distinction is between Parts 3–5 (the IPVS table) and Parts 6–8 (the Keepalived layer that makes the virtual IP highly available).

Quick Reference

Load-Balancing Layers

Layer	Tool	Routing Decision
Layer 4	IPVS (kernel)	IP, port, protocol
Layer 7	HAProxy, Nginx	URL, headers, cookies

IPVS Algorithms

Algorithm	Directive	Use Case
Round-robin	`-s rr`	Equal backends, uniform connections
Weighted round-robin	`-s wrr`	Unequal capacity
Least-connection	`-s lc`	Variable connection duration
Weighted least-connection	`-s wlc`	Default; unequal capacity and variable duration
Source hash	`-s sh`	Session persistence

IPVS Routing Methods

Method	Flag	Response Path
NAT	`-m`	Through the load balancer
Direct Routing	`-g`	Directly to the client
Tunneling	`-i`	Directly to the client

Health Check and HA

Component	Purpose
HAProxy `option httpchk`	HTTP health check
HAProxy `check`	Enable check on server
Keepalived `TCP_CHECK`	TCP health check for IPVS
Keepalived `vrrp_instance`	Floating IP failover
Keepalived `virtual_server`	IPVS service definition

Best Practices

✅ Do This:

# Use weighted round-robin for unequal backends
ipvsadm -A -t 192.168.1.100:80 -s wrr                          # ✅

# Use Direct Routing for high throughput
ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.10:80 -g          # ✅

# Configure health checks
option httpchk GET /health                                      # ✅

# Use Keepalived for the virtual IP
vrrp_instance VI_1 { virtual_ipaddress { 192.168.1.100/24 } }  # ✅

# Use least-connection for variable request durations
balance leastconn                                              # ✅

❌ Don’t Do This:

# Use NAT mode for a large cluster
# The load balancer becomes the bottleneck                       # ❌

# Omit health checks
# Failed backends stay in rotation                              # ❌

# Rely on a single load balancer without VRRP
# The load balancer is a single point of failure                # ❌

# Use round-robin for long-lived connections
# One backend receives most of the connections                  # ❌

# Use IP-hash without considering NAT
# Multiple clients behind a NAT share one IP                    # ❌

Common Pitfalls

Pitfall	Why It Happens	Fix
NAT bottleneck	All response traffic passes through the load balancer	Use Direct Routing or Tunneling
Failed backend stays in rotation	No health check configured	Add `TCP_CHECK` or `option httpchk`
VIP does not fail over	Keepalived not running on the backup	Start Keepalived on both nodes
Session loss	No session persistence	Use IP-hash or a shared session store
Backend does not respond to VIP in DR mode	ARP not suppressed on the backend	Configure `arp_ignore` and `arp_announce`

Real-World Examples

1. IPVS round-robin

ipvsadm -A -t 192.168.1.100:80 -s rr

2. IPVS weighted round-robin

ipvsadm -A -t 192.168.1.100:80 -s wrr

3. IPVS NAT mode

ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.10:80 -m

4. IPVS Direct Routing

ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.10:80 -g

5. HAProxy round-robin

balance roundrobin

6. HAProxy least-connection

balance leastconn

7. HAProxy health check

option httpchk GET /health

8. Keepalived VRRP

vrrp_instance VI_1 { virtual_ipaddress { 192.168.1.100/24 } }

9. Keepalived virtual server

virtual_server 192.168.1.100 80 { lb_algo rr; lb_kind NAT }

10. Verify IPVS table

ipvsadm -Ln

Visual

Layer 4 vs Layer 7

┌──────────────────────────────────────────────────────────────┐
│  LAYER 4 FORWARDS PACKETS; LAYER 7 PARSES REQUESTS           │
│                                                              │
│  Layer 4 (IPVS):                                             │
│  ┌────────────────────────────────────────────────────────┐  │
│  │ Client → Load Balancer → Backend                      │  │
│  │                                                        │  │
│  │ Decision based on: IP, port, protocol                  │  │
│  │ Does not inspect the payload                           │  │
│  │ Very high throughput (kernel space)                    │  │
│  │ Any TCP/UDP protocol                                   │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  Layer 7 (HAProxy, Nginx):                                   │
│  ┌────────────────────────────────────────────────────────┐  │
│  │ Client → Proxy → Backend                               │  │
│  │                                                        │  │
│  │ Decision based on: URL, headers, cookies               │  │
│  │ Parses the HTTP request                                │  │
│  │ Lower throughput (userspace)                           │  │
│  │ HTTP/HTTPS routing rules                               │  │
│  └────────────────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────┘

IPVS Routing Methods

┌──────────────────────────────────────────────────────────────┐
│  THREE METHODS FOR DELIVERING PACKETS                        │
│                                                              │
│  NAT:                                                        │
│  Client → LB → Backend                                       │
│  Backend → LB → Client                                       │
│  (all traffic through the LB; bottleneck)                    │
│                                                              │
│  Direct Routing:                                             │
│  Client → LB → Backend                                       │
│  Backend → Client                                            │
│  (response bypasses the LB; scalable)                        │
│                                                              │
│  Tunneling:                                                  │
│  Client → LB → (tunnel) → Backend                            │
│  Backend → Client                                            │
│  (response bypasses the LB; backends on different networks)  │
└──────────────────────────────────────────────────────────────┘

High Availability with Keepalived

┌──────────────────────────────────────────────────────────────┐
│  VRRP MOVES THE VIRTUAL IP BETWEEN NODES                     │
│                                                              │
│  Primary (MASTER, priority 100):                             │
│  ┌────────────────────────────────────────────────────────┐  │
│  │ eth0: 192.168.1.10                                     │  │
│  │ VIP:  192.168.1.100  ← active                          │  │
│  │ IPVS table with real servers                           │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  Backup (BACKUP, priority 90):                               │
│  ┌────────────────────────────────────────────────────────┐  │
│  │ eth0: 192.168.1.11                                     │  │
│  │ VIP:  (not assigned)                                   │  │
│  │ IPVS table standby                                     │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  Primary fails → VIP moves to backup → clients reconnect     │
└──────────────────────────────────────────────────────────────┘

Summary

Item	Value
Layer 4 tool	IPVS (kernel)
Layer 7 tools	HAProxy, Nginx
Round-robin	`-s rr` / `balance roundrobin`
Weighted round-robin	`-s wrr` / `balance roundrobin` with weights
Least-connection	`-s lc` / `balance leastconn`
Source hash	`-s sh` / `balance source`
NAT mode	`-m`; all traffic through LB
Direct Routing	`-g`; response bypasses LB
Tunneling	`-i`; response bypasses LB
Health checks	`TCP_CHECK`, `option httpchk`
High availability	Keepalived + VRRP

Key takeaways:

Layer 4 load balancing distributes connections; Layer 7 distributes requests. IPVS operates on IP, port, and protocol, and runs in the kernel for high throughput. HAProxy and Nginx parse the application protocol and can route based on URL, headers, or cookies .
The routing method determines the response path. NAT sends all traffic through the load balancer, which becomes a bottleneck. Direct Routing and Tunneling send the response directly from the backend to the client, which allows the cluster to scale .
The algorithm must match the workload. Round-robin works for uniform connections; least-connection works for variable durations; weighted algorithms work for unequal backend capacity; source hash provides session persistence for applications that store state locally .
Health checks remove failed backends. HAProxy’s option httpchk and Keepalived’s TCP_CHECK detect failures and remove the backend from rotation. A backend that passes the check again is returned to the pool.
The load balancer must itself be highly available. Keepalived uses VRRP to move a virtual IP between a primary and a backup node. The virtual IP is the address that clients connect to, and it moves to the backup when the primary fails .
Direct Routing requires ARP suppression on the backends. The backends must accept packets for the virtual IP without responding to ARP requests for it. This is configured with arp_ignore and arp_announce on the backend interfaces .

Remember: Load balancing is a layered design. The algorithm selects the backend. The routing method determines how the response returns. The health check removes failed backends. The high-availability layer ensures that the load balancer itself does not become a single point of failure. Use IPVS for Layer 4 throughput, HAProxy or Nginx for Layer 7 routing, and Keepalived for the virtual IP. The choice of algorithm and routing method depends on the workload and the topology, not on preference.

Was this helpful?

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!