LFCE 61 🐧 Load Balancing with Nginx - KandZ

⏱️ Estimated read: 12 min | |

Nginx is the most common software load balancer for HTTP traffic on Linux. The previous chapter covered the concepts—the algorithms, the health checks, and the high-availability layer. This chapter covers the Nginx configuration that implements them: the upstream block that defines the backend pool, the algorithm directive that selects the distribution method, the passive health-check parameters that remove failed backends from rotation, and the keepalive settings that reduce the connection overhead between the proxy and the backends.

The configuration is built from two layers. The upstream block defines the pool, the algorithm, and the health-check parameters. The server block with proxy_pass forwards the requests to the pool. The upstream block is the only place where the load-balancing method is configured; the proxy_pass directive simply references the pool by name .

Key point: The default algorithm is round-robin. The least_conn directive selects the backend with the fewest active connections, which is appropriate when request durations vary. The ip_hash directive maps the client’s IP address to a backend for session persistence . Passive health checks are configured with max_fails and fail_timeout on each server line; a backend that fails the threshold is marked unavailable for the timeout period .

Why Nginx’s upstream block requires explicit configuration

The algorithm-selection problem.

Nginx does not choose the algorithm for the workload. The default round-robin distributes requests evenly, which works when the backends have similar capacity and the requests complete in similar time. If one backend is slower, round-robin continues to send it the same share of requests, and its active connection count grows. The least_conn directive addresses this by selecting the backend with the fewest active connections, so the slower backend receives fewer new requests until it catches up . The ip_hash directive is the workaround for applications that store session state locally, but it is less reliable than a shared session store.

The health-check problem.

Nginx’s open-source edition does not have active health checks. It has passive health checks: it observes the responses to real requests and marks a backend as failed when it returns connection errors or timeouts. The max_fails parameter is the number of failures within the fail_timeout window that trigger the unavailable state. The fail_timeout parameter is both the window for counting failures and the duration for which the backend is marked unavailable . Without these parameters, Nginx uses the default of one failure and ten seconds, which may be too aggressive or too lenient for the workload.

The connection-overhead problem.

By default, Nginx opens a new connection to the backend for every request. On a high-traffic proxy, the TCP handshake and the connection teardown consume resources on both the proxy and the backend. The keepalive directive in the upstream block maintains a pool of idle connections that are reused for subsequent requests. The keepalive directive requires proxy_http_version 1.1 and an empty Connection header in the location block; without them, the keepalive pool is not used .

a. The upstream block and the algorithms

The upstream block defines the backend pool. The algorithm directive is placed at the top of the block, and the server lines define the backends.

# /etc/nginx/conf.d/loadbalancer.conf
upstream backend {
    # Default: round-robin
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
    server 10.0.0.3:8080;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

The proxy_pass http://backend; directive references the pool by name. Nginx distributes the requests according to the algorithm. The default is round-robin, which sends each request to the next backend in the sequence .

The least_conn directive changes the algorithm to least-connections. It is placed at the top of the upstream block, before the server lines.

upstream backend {
    least_conn;
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
}

The least_conn algorithm considers the number of active connections on each backend and selects the one with the fewest. When multiple backends have the same number, it uses weighted round-robin among them . This is the correct choice when the backends have similar capacity but the request durations vary, because it avoids sending new requests to a backend that is already processing several slow requests.

The ip_hash directive provides session persistence by hashing the client’s IP address. It is placed at the top of the upstream block.

upstream backend {
    ip_hash;
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
}

The ip_hash algorithm maps the client’s IP to a backend, so requests from the same client always reach the same backend unless it is unavailable . This is a workaround for applications that store session state locally. The better solution is a shared session store, but ip_hash is valid when the application cannot be changed.

Weights can be added to any algorithm. A backend with weight=3 receives three times as many requests as a backend with the default weight of 1. The weights are supported with round-robin, least-connections, and ip-hash in recent Nginx versions .

upstream backend {
    server 10.0.0.1:8080 weight=3;
    server 10.0.0.2:8080 weight=2;
    server 10.0.0.3:8080 weight=1;
}

b. Passive health checks and failover

The passive health checks are configured on each server line with the max_fails and fail_timeout parameters. A backend that fails the threshold is marked unavailable and removed from rotation.

upstream backend {
    least_conn;
    server 10.0.0.1:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.2:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.3:8080 backup;
}

The max_fails=3 parameter means that three failed attempts within the fail_timeout window mark the backend as unavailable. The fail_timeout=30s parameter is both the window for counting the failures and the duration for which the backend is marked unavailable . After the thirty seconds expire, Nginx sends a single request to probe the backend; if it succeeds, the backend is returned to rotation.

The backup parameter marks the third backend as a standby. It receives traffic only when all the primary backends are unavailable. This is useful for a disaster-recovery server that does not need to handle normal traffic .

The down parameter marks a backend as administratively disabled. It is used during maintenance to remove a backend from rotation without deleting its configuration.

upstream backend {
    server 10.0.0.1:8080;
    server 10.0.0.2:8080 down;   # Temporarily disabled for maintenance
}

c. Keepalive connections to the backends

The keepalive directive in the upstream block maintains a pool of idle connections that are reused for subsequent requests. It reduces the connection overhead on both the proxy and the backends.

upstream backend {
    least_conn;
    server 10.0.0.1:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.2:8080 max_fails=3 fail_timeout=30s;

    keepalive 32;              # Maintain 32 idle connections
    keepalive_timeout 60s;     # Idle connection timeout
    keepalive_requests 100;    # Max requests per connection
}

server {
    location / {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
    }
}

The keepalive 32 directive tells Nginx to maintain up to thirty-two idle connections per worker process. The keepalive_timeout and keepalive_requests directives control how long an idle connection is kept and how many requests it can serve before it is closed .

The proxy_http_version 1.1 and proxy_set_header Connection ""; directives in the location block are required for the keepalive pool to be used. The HTTP/1.1 protocol supports persistent connections, and the empty Connection header removes the default close directive that Nginx sends. Without these two directives, Nginx closes the connection after each request, and the keepalive pool is bypassed .

Complete Example Session

# ============================================
# PART 1: START TEST BACKENDS
# ============================================
python3 -m http.server 8080 --directory /var/www/backend1 &
python3 -m http.server 8081 --directory /var/www/backend2 &
python3 -m http.server 8082 --directory /var/www/backend3 &

# ============================================
# PART 2: CREATE THE LOAD BALANCER CONFIGURATION
# ============================================
sudo tee /etc/nginx/conf.d/loadbalancer.conf > /dev/null <<'EOF'
upstream backend {
    least_conn;
    server 127.0.0.1:8080 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:8081 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:8082 backup;

    keepalive 32;
    keepalive_timeout 60s;
    keepalive_requests 100;
}

server {
    listen 80;
    server_name lb.local;

    location / {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}
EOF

# ============================================
# PART 3: VALIDATE AND RELOAD
# ============================================
sudo nginx -t
sudo systemctl reload nginx

# ============================================
# PART 4: TEST LOAD BALANCING
# ============================================
for i in {1..10}; do curl -s -H "Host: lb.local" http://localhost; done
# The responses alternate between backend1 and backend2

# ============================================
# PART 5: TEST FAILOVER
# ============================================
kill %1    # Stop the first backend
curl -I -H "Host: lb.local" http://localhost
# Nginx marks the backend as failed after the threshold
# and sends the request to the second backend

# ============================================
# PART 6: TEST BACKUP SERVER
# ============================================
kill %2    # Stop the second backend
curl -I -H "Host: lb.local" http://localhost
# The backup backend receives the request

# ============================================
# PART 7: CLEAN UP
# ============================================
kill %1 %2 %3
sudo rm /etc/nginx/conf.d/loadbalancer.conf
sudo systemctl reload nginx

The seven parts demonstrate the load balancer configuration: the test backends, the upstream block with least_conn, the passive health checks, the keepalive pool, the validation, the load balancing test, and the failover test. The critical verification is Part 5: when a backend is stopped, Nginx marks it as failed and routes the requests to the remaining backends.

Quick Reference

Load-Balancing Algorithms

Directive	Algorithm	Use Case
(default)	Round-robin	Equal backends, uniform requests
`least_conn`	Least connections	Variable request duration
`ip_hash`	IP hash	Session persistence

Server Parameters

Parameter	Purpose	Example
`weight=N`	Request share	`weight=3`
`max_fails=N`	Failures before unavailable	`max_fails=3`
`fail_timeout=Ns`	Failure window and retry interval	`fail_timeout=30s`
`backup`	Standby server	`backup`
`down`	Administratively disabled	`down`

Keepalive Directives

Directive	Purpose
`keepalive N`	Idle connections per worker
`keepalive_timeout`	Idle connection timeout
`keepalive_requests`	Max requests per connection
`proxy_http_version 1.1`	Required for keepalive
`proxy_set_header Connection ""`	Required for keepalive

Best Practices

✅ Do This:

# Use least_conn for variable request durations
upstream backend {
    least_conn;
    server 10.0.0.1:8080;
}                                                            # ✅

# Configure passive health checks
server 10.0.0.1:8080 max_fails=3 fail_timeout=30s;           # ✅

# Use a backup server for failover
server 10.0.0.3:8080 backup;                                 # ✅

# Enable keepalive for connection reuse
keepalive 32;                                                # ✅

# Include the required keepalive directives
proxy_http_version 1.1;
proxy_set_header Connection "";                              # ✅

# Validate before reloading
nginx -t                                                      # ✅

❌ Don’t Do This:

# Use round-robin for variable request durations
upstream backend { server 10.0.0.1:8080; }    # Uneven load   # ❌

# Omit health checks
server 10.0.0.1:8080;    # Failed backend stays in rotation   # ❌

# Use ip_hash behind a NAT
# Multiple clients share one IP and one backend              # ❌

# Enable keepalive without the required directives
keepalive 32;    # The pool is not used without them          # ❌

# Use least_time or health_check on open source
# These are Nginx Plus only                                  # ❌

Common Pitfalls

Pitfall	Why It Happens	Fix
Uneven load distribution	Round-robin with variable requests	Use `least_conn`
Failed backend stays in rotation	`max_fails` and `fail_timeout` not set	Add the parameters
Keepalive not used	Missing `proxy_http_version 1.1`	Add the directives
Session loss	Round-robin without persistence	Use `ip_hash` or a shared session store
Backup never receives traffic	Primary backends always available	Expected; `backup` is for failover
`least_time` fails `nginx -t`	Nginx Plus directive on open source	Use `least_conn`

Real-World Examples

1. Round-robin

upstream backend { server 10.0.0.1:8080; server 10.0.0.2:8080; }

2. Least-connections

upstream backend { least_conn; server 10.0.0.1:8080; server 10.0.0.2:8080; }

3. IP hash

upstream backend { ip_hash; server 10.0.0.1:8080; server 10.0.0.2:8080; }

4. Weighted

server 10.0.0.1:8080 weight=3;

5. Passive health check

server 10.0.0.1:8080 max_fails=3 fail_timeout=30s;

6. Backup server

server 10.0.0.3:8080 backup;

7. Keepalive pool

upstream backend { keepalive 32; server 10.0.0.1:8080; }

8. Keepalive directives

proxy_http_version 1.1;
proxy_set_header Connection "";

9. Validate configuration

nginx -t

10. Test load balancing

for i in {1..10}; do curl -s -H "Host: lb.local" http://localhost; done

Visual

Nginx Load Balancer Architecture

┌──────────────────────────────────────────────────────────────┐
│  CLIENT → NGINX UPSTREAM → BACKENDS                          │
│                                                              │
│  Client                                                      │
│  ┌─────────┐                                                 │
│  │ Browser │                                                 │
│  └────┬────┘                                                 │
│       │ HTTP                                                 │
│       ▼                                                      │
│  ┌────────────────────────────────────────────────────────┐  │
│  │ Nginx                                                  │  │
│  │                                                        │  │
│  │ upstream backend {                                     │  │
│  │   least_conn;                                          │  │
│  │   server 10.0.0.1:8080 max_fails=3 fail_timeout=30s;   │  │
│  │   server 10.0.0.2:8080;                                │  │
│  │   keepalive 32;                                        │  │
│  │ }                                                      │  │
│  └──────────────────────────┬─────────────────────────────┘  │
│                             │                                │
│              ┌──────────────┴──────────────┐                 │
│              ▼                             ▼                 │
│  ┌─────────────────┐            ┌─────────────────┐          │
│  │ Backend 1       │            │ Backend 2       │          │
│  │ :8080           │            │ :8080           │          │
│  └─────────────────┘            └─────────────────┘          │
└──────────────────────────────────────────────────────────────┘

Passive Health Check

┌──────────────────────────────────────────────────────────────┐
│  FAILED BACKENDS ARE MARKED UNAVAILABLE                      │
│                                                              │
│  server 10.0.0.1:8080 max_fails=3 fail_timeout=30s;          │
│                                                              │
│  Backend 1 fails 3 requests within 30 seconds:               │
│  → Marked unavailable                                        │
│  → Removed from rotation for 30 seconds                      │
│  → After 30s, one request probes it                          │
│     ├── Success → returned to rotation                       │
│     └── Failure → marked unavailable for another 30s         │
└──────────────────────────────────────────────────────────────┘

Keepalive Connection Reuse

┌──────────────────────────────────────────────────────────────┐
│  KEEPALIVE REDUCES CONNECTION OVERHEAD                       │
│                                                              │
│  Without keepalive:                                          │
│  Request 1 → new connection → backend                        │
│  Request 2 → new connection → backend                        │
│  Request 3 → new connection → backend                        │
│  (TCP handshake for every request)                           │
│                                                              │
│  With keepalive:                                             │
│  Request 1 → new connection → backend                        │
│  Request 2 → reused connection                               │
│  Request 3 → reused connection                               │
│  (no handshake for subsequent requests)                      │
└──────────────────────────────────────────────────────────────┘

Summary

Item	Value
Upstream block	Defines the backend pool
Default algorithm	Round-robin
Least-connections	`least_conn`
Session persistence	`ip_hash`
Weight	`weight=N`
Passive health check	`max_fails`, `fail_timeout`
Backup server	`backup`
Maintenance mode	`down`
Keepalive pool	`keepalive N`
Required keepalive directives	`proxy_http_version 1.1`, `Connection ""`
Validation	`nginx -t`

Key takeaways:

The upstream block defines the pool and the algorithm. The proxy_pass directive references the pool by name. The default algorithm is round-robin, and least_conn and ip_hash change the distribution method .
least_conn is better for variable request durations. It selects the backend with the fewest active connections, so a slower backend receives fewer new requests until it catches up. Round-robin continues to send it the same share regardless of its load.
Passive health checks remove failed backends. The max_fails and fail_timeout parameters define the threshold and the retry interval. A backend that fails is marked unavailable and removed from rotation for the timeout period .
The backup parameter provides failover. A backup backend receives traffic only when all the primary backends are unavailable. The down parameter disables a backend for maintenance without removing its configuration.
Keepalive reduces connection overhead. The keepalive directive maintains a pool of idle connections. The proxy_http_version 1.1 and proxy_set_header Connection ""; directives are required for the pool to be used .
The open-source edition does not have active health checks. The least_time and health_check directives are Nginx Plus only. The open-source alternative is passive health checks or a third-party module .

Remember: Nginx load balancing is configured in the upstream block. The algorithm directive determines the distribution, the server lines define the backends and the health-check parameters, and the keepalive directives reduce the connection overhead. The proxy_pass directive references the pool. Validate with nginx -t, reload with systemctl reload nginx, and test with curl using the correct Host header. The load balancer is only as good as its health checks and its algorithm choice; the default round-robin is not always the correct choice.

Was this helpful?

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!