REST API 25 🌐 API Gateways, Reverse Proxies, and Load Balancing
The three components that sit between clients and backend services are often conflated because their data paths overlap. A reverse proxy accepts a request, selects an upstream, forwards it, and relays the response. A load balancer distributes traffic across multiple backends. An API gateway performs reverse-proxy work while adding API-specific policy: consumer identity, per-route authorization, rate limits, and usage analytics . The distinction is not about protocol boundaries—any of these can be configured to do parts of the others’ jobs. It is about scope and operating model. A reverse proxy is infrastructure. A load balancer is distribution. An API gateway is a shared policy layer for APIs and their consumers.
For the REST API series, the practical question is not “which one is better” but “which concern belongs at which layer.” Stacking all three without distinct responsibilities adds latency without adding safety. The canonical production edge separates them: a Layer 4 load balancer for raw distribution, an API gateway for per-route policy, and a service mesh for east-west security .
Key point: Use a reverse proxy when you need TLS termination and path routing for one or a few services. Use an API gateway when you need per-consumer authentication, quotas, and transformations across many services. Use a load balancer when the primary need is distributing identical replicas.
Why these components exist
The exposure problem. Backend services should not be directly reachable from the internet. A reverse proxy presents a single public hostname and hides internal topology. It terminates TLS, buffers slow clients, and can serve static assets without touching the application servers . This is the simplest and most common use of NGINX: a wall between the public network and the services behind it.
The scale problem. A single backend instance has a ceiling. A load balancer distributes requests across multiple identical replicas, removes unhealthy instances from rotation, and keeps the service available when one node fails . The distribution algorithm matters: round-robin for uniform requests, least-connections for long-lived or variable-duration requests, and IP hash when session persistence is required without a shared session store .
The API governance problem. When ten services sit behind one public hostname, the platform team needs a place to enforce policies that should not be reimplemented in every service: API key validation, per-tenant rate limits, request transformation, and usage attribution. An API gateway provides that place . The same technical control—a rate limit—has a different operational role when it is attached to a consumer plan and managed across hundreds of routes.
The redundancy problem. Each proxy hop adds latency. A gateway adds roughly 5 to 9 milliseconds at p99 compared with a plain reverse proxy . If a request passes through an L4 load balancer, an L7 load balancer, a reverse proxy, and a service mesh sidecar, the cumulative overhead becomes measurable. The anti-pattern is stacking layers without assigning each one a distinct responsibility .
a. Reverse Proxy and Load Balancing with NGINX
NGINX is the most common implementation of both roles. A reverse proxy configuration forwards requests to a backend. Adding an upstream block with multiple servers turns the same configuration into a load balancer .
# Reverse proxy with load balancing across three backends
http {
upstream backend_servers {
# Default: round-robin across all servers
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;
}
server {
listen 443 ssl;
server_name api.example.com;
ssl_certificate /etc/nginx/certs/fullchain.pem;
ssl_certificate_key /etc/nginx/certs/privkey.pem;
location / {
proxy_pass http://backend_servers;
# Pass the original client IP to the backend
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
}
}
}The load balancing algorithm is selected in the upstream block. Round-robin is the default. least_conn prefers the server with the fewest active connections, which is better when request duration varies. ip_hash routes the same client IP to the same server, providing session persistence without a shared store .
upstream backend_servers {
# Least-connections: better for long-lived requests
least_conn;
server backend1.example.com;
server backend2.example.com;
}
upstream sticky_servers {
# IP hash: same client IP always reaches the same backend
ip_hash;
server backend1.example.com;
server backend2.example.com;
}Passive health checks are available in open-source NGINX. A server that fails max_fails times within fail_timeout is marked unavailable and removed from rotation until the timeout expires .
upstream backend_servers {
# Mark down after 3 failures within 30 seconds
server backend1.example.com max_fails=3 fail_timeout=30s;
server backend2.example.com max_fails=3 fail_timeout=30s;
}The reverse proxy role and the load balancer role overlap in NGINX. A single server block that references an upstream group with multiple servers is doing both: it terminates TLS, forwards requests, and distributes them across backends . The distinction matters when deciding whether to add a dedicated gateway for policy that NGINX is not configured to handle.
b. API Gateway Scope and When to Use It
An API gateway is a reverse proxy with an API-consumer model. It authenticates the caller, applies policy per route or per consumer, transforms requests and responses, and emits telemetry attributed to an API key or tenant . The gateway does not replace service-level authorization. A service must still decide whether an authenticated user can perform a specific action on a specific resource .
The gateway is the right tool when a public API has many consumers and many routes. It centralizes the cross-cutting concerns that would otherwise be duplicated: JWT validation, API key lookup, quota enforcement, and request logging with consumer attribution . It is the wrong tool for a single backend that needs TLS termination. For that case, a plain reverse proxy costs less in latency and operational complexity .
# Conceptual gateway policy: per-route auth and rate limit
routes:
- path: /api/orders
methods: [GET, POST]
auth: jwt
rate_limit:
per_consumer: 1000/minute
upstream: orders-service
- path: /api/public/status
methods: [GET]
auth: none
rate_limit:
per_ip: 10/minute
upstream: status-serviceThe gateway’s value is the control plane, not the data path. The data path is a reverse proxy. The control plane is where policies are defined, versioned, and applied across environments. A gateway without a consumer model—just a proxy with rate limiting—does not justify its overhead .
c. Layer Separation and the Anti-Pattern
The most common architectural mistake is stacking components without assigning distinct responsibilities. An L4 load balancer for DDoS absorption, an L7 load balancer for path routing, a reverse proxy for TLS termination, and a service mesh sidecar for mTLS all in one request path adds 1 to 3 milliseconds per hop without adding safety if their jobs overlap .
The canonical separation assigns one job to each layer. The L4 load balancer distributes connections across gateway instances. The API gateway applies consumer policy. The service mesh handles service-to-service mTLS and retries . If a distributed trace shows proxy hops consuming more than 20 percent of end-to-end latency, there are redundant layers .
Client
│
▼
┌─────────────────┐
│ L4 Load Balancer│ Distributes TCP connections
│ (e.g., NLB) │ DDoS absorption
└────────┬────────┘
▼
┌─────────────────┐
│ API Gateway │ Per-route auth, quotas
│ (e.g., Kong) │ Consumer identity
└────────┬────────┘
▼
┌─────────────────┐
│ Service Mesh │ East-west mTLS
│ (e.g., Envoy) │ Retries, circuit breaking
└────────┬────────┘
▼
Backend ServicesThe gateway should not become the only authorization layer. Object-level and function-level authorization remain the service’s responsibility . The gateway verifies identity and applies coarse-grained policy. The service enforces fine-grained business rules.
Complete Example Session
# ============================================
# PART 1: REVERSE PROXY ONLY
# ============================================
# One backend, TLS termination, header forwarding
server {
listen 443 ssl;
server_name app.example.com;
ssl_certificate /etc/nginx/certs/app.pem;
ssl_certificate_key /etc/nginx/certs/app.key;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
# ============================================
# PART 2: ADD LOAD BALANCING
# ============================================
# Three identical backends, round-robin
upstream app_backends {
server 10.0.0.10:3000;
server 10.0.0.11:3000;
server 10.0.0.12:3000;
}
server {
listen 443 ssl;
server_name app.example.com;
# ... TLS config ...
location / {
proxy_pass http://app_backends;
}
}
# ============================================
# PART 3: LEAST-CONNECTIONS ALGORITHM
# ============================================
# Better when request duration varies
upstream app_backends {
least_conn;
server 10.0.0.10:3000;
server 10.0.0.11:3000;
}
# ============================================
# PART 4: SESSION PERSISTENCE WITH IP HASH
# ============================================
# Same client IP reaches the same backend
upstream app_backends {
ip_hash;
server 10.0.0.10:3000;
server 10.0.0.11:3000;
}
# ============================================
# PART 5: PASSIVE HEALTH CHECKS
# ============================================
# Remove failing servers from rotation
upstream app_backends {
server 10.0.0.10:3000 max_fails=3 fail_timeout=30s;
server 10.0.0.11:3000 max_fails=3 fail_timeout=30s;
}
# ============================================
# PART 6: PATH-BASED ROUTING
# ============================================
# Different paths to different upstreams
upstream orders_backend { server 10.0.1.10:8080; }
upstream users_backend { server 10.0.2.10:8080; }
server {
listen 443 ssl;
server_name api.example.com;
location /api/orders/ {
proxy_pass http://orders_backend;
}
location /api/users/ {
proxy_pass http://users_backend;
}
}
# ============================================
# PART 7: ADD CACHING AT THE PROXY
# ============================================
# Cache GET responses to reduce backend load
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=api_cache:10m max_size=1g;
server {
# ...
location /api/public/ {
proxy_cache api_cache;
proxy_cache_valid 200 10m;
add_header X-Proxy-Cache $upstream_cache_status;
proxy_pass http://public_backend;
}
}
# ============================================
# PART 8: GATEWAY-STYLE AUTH CHECK (CONCEPTUAL)
# ============================================
# NGINX can validate JWTs via auth_request to an auth service
location /api/private/ {
auth_request /_auth;
proxy_pass http://private_backend;
}
location = /_auth {
internal;
proxy_pass http://auth-service:9000/validate;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
}
# ============================================
# PART 9: RATE LIMITING AT THE EDGE
# ============================================
# Simple per-IP rate limit
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
server {
location /api/ {
limit_req zone=api_limit burst=20 nodelay;
proxy_pass http://app_backends;
}
}
# ============================================
# PART 10: DECISION CHECKLIST
# ============================================
# - One service, TLS only? → Reverse proxy
# - Multiple identical replicas? → Add upstream load balancing
# - Many services, per-consumer auth? → API gateway
# - Full stack? → L4 LB → Gateway → Mesh
# - Trace shows 20%+ proxy latency? → Remove redundant layerThe ten parts move from a plain reverse proxy through load balancing algorithms, health checks, path routing, caching, auth delegation, and rate limiting. The decision at the end is the architectural question: each added layer must justify its latency with a responsibility that no existing layer covers.
Quick Reference
Component Comparison
| Component | Primary Job | Scope | Typical Tool |
|---|---|---|---|
| Reverse proxy | Forward and secure traffic | Infrastructure | NGINX, HAProxy |
| Load balancer | Distribute across replicas | Availability | NLB, ALB, NGINX upstream |
| API gateway | Govern API consumers | Policy and identity | Kong, APISIX, Zuul |
NGINX Load Balancing Algorithms
| Algorithm | Directive | Use When |
|---|---|---|
| Round-robin | (default) | Requests are uniform |
| Least connections | least_conn | Request duration varies |
| IP hash | ip_hash | Session persistence needed |
| Weighted | weight=N | Servers have unequal capacity |
Gateway versus Reverse Proxy
| Concern | Reverse Proxy | API Gateway |
|---|---|---|
| TLS termination | Yes | Yes |
| Path routing | Yes | Yes |
| Load balancing | Yes | Yes |
| Per-consumer API keys | No | Yes |
| Usage quotas | No | Yes |
| Request transformation | Limited | Native |
| Consumer-level telemetry | No | Yes |
Best Practices
✅ Do This:
# Use least_conn for variable-duration requests
upstream backend { least_conn; server a; server b; } # ✅# Set max_fails and fail_timeout for passive health checks
server backend1 max_fails=3 fail_timeout=30s; # ✅# Pass original client IP and host to backends
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host; # ✅# Assign one responsibility per layer
L4 for distribution, gateway for policy, mesh for east-west # ✅# Use a gateway only when per-consumer policy is required
API keys, quotas, usage plans across many services # ✅# Measure proxy latency in distributed traces
If hops consume >20% of end-to-end, remove redundancy # ✅❌ Don’t Do This:
# Stack proxies without distinct jobs
NLB → ALB → NGINX → Envoy sidecar in one path # ❌# Use round-robin for long-lived requests
# Causes imbalance when one request holds a connection # ❌# Use an API gateway for a single service
Paying 5-9ms p99 for features you do not use # ❌# Forget to forward client IP
# Backend logs show proxy IP for every request # ❌# Make the gateway the only authorization layer
Services still need object-level checks # ❌# Use L4 balancing for gRPC
HTTP/2 multiplexes streams over one connection; L4 pins them all # ❌Common Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
| Session lost between requests | Round-robin sends client to different backends | Use ip_hash or a shared session store |
| Failing backend still receives traffic | No health checks configured | Add max_fails/fail_timeout |
| Proxy latency dominates trace | Too many redundant layers | Remove overlapping proxies |
| Gateway adds 5ms for no benefit | Used for single service | Use reverse proxy instead |
| Backend sees proxy IP | X-Forwarded-For not set | Forward client IP headers |
| gRPC traffic imbalanced | L4 balancer pins HTTP/2 streams to one backend | Use L7 balancer |
Real-World Examples
1. Basic Reverse Proxy
location / { proxy_pass http://127.0.0.1:3000; }2. Round-Robin Load Balancing
upstream backend { server a; server b; server c; }3. Least-Connections
upstream backend { least_conn; server a; server b; }4. Session Persistence
upstream backend { ip_hash; server a; server b; }5. Passive Health Check
server a max_fails=3 fail_timeout=30s;6. Path-Based Routing
location /api/orders/ { proxy_pass http://orders; }7. Proxy Caching
proxy_cache api_cache; proxy_cache_valid 200 10m;8. Rate Limiting at Edge
limit_req zone=api_limit burst=20 nodelay;9. Forward Client IP
proxy_set_header X-Real-IP $remote_addr;10. Decision Check
1 service → reverse proxy. 10+ services → gateway.Visual
The Overlap and the Distinction
┌──────────────────────────────────────────────────────────────┐
│ ALL THREE FORWARD TRAFFIC │
│ │
│ Client ──► Component ──► Backend │
│ │
│ ┌─────────────────┐ │
│ │ Reverse Proxy │ Accepts request, selects upstream, │
│ │ │ forwards, returns response │
│ └─────────────────┘ │
│ │
│ ┌─────────────────┐ │
│ │ Load Balancer │ Same path, plus distribution across │
│ │ │ multiple identical backends │
│ └─────────────────┘ │
│ │
│ ┌─────────────────┐ │
│ │ API Gateway │ Same path, plus consumer identity, │
│ │ │ per-route policy, quotas, analytics │
│ └─────────────────┘ │
│ │
│ The difference is scope, not protocol. │
│ Any of the three can be configured to do parts of the │
│ others' jobs. The question is what responsibility the │
│ layer owns. │
│ │
└──────────────────────────────────────────────────────────────┘NGINX Upstream and Proxy Relationship
┌──────────────────────────────────────────────────────────────┐
│ server block (reverse proxy role) │
│ │
│ ┌─────────────────────────────────────────┐ │
│ │ listen 443 ssl │ │
│ │ server_name api.example.com │ │
│ │ │ │
│ │ location / { │ │
│ │ proxy_pass http://backend_group; │ │
│ │ } │ │
│ └──────────────────┬──────────────────────┘ │
│ │ │
│ ▼ │
│ upstream block (load balancer role) │
│ ┌─────────────────────────────────────────┐ │
│ │ upstream backend_group { │ │
│ │ least_conn; │ │
│ │ server 10.0.0.10:3000; │ │
│ │ server 10.0.0.11:3000; │ │
│ │ } │ │
│ └─────────────────────────────────────────┘ │
│ │
│ One server block, one upstream block. │
│ The same configuration performs both roles. │
│ Adding servers to upstream turns proxy into load balancer. │
│ │
└──────────────────────────────────────────────────────────────┘The Canonical Layer Separation
┌──────────────────────────────────────────────────────────────┐
│ CLIENT │
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ L4 LOAD BALANCER│ Job: distribute TCP connections │
│ │ (NLB, Unimog) │ DDoS absorption, source-IP preservation│
│ └────────┬────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ API GATEWAY │ Job: per-route policy │
│ │ (Kong, Zuul) │ Auth, quotas, transformations │
│ └────────┬────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ SERVICE MESH │ Job: east-west security │
│ │ (Envoy, Istio) │ mTLS, retries, circuit breaking │
│ └────────┬────────┘ │
│ │ │
│ ▼ │
│ Backend Services │
│ │
│ Each layer owns one concern. │
│ If a layer duplicates another's job, remove it. │
│ │
└──────────────────────────────────────────────────────────────┘Latency Cost of Redundant Layers
┌──────────────────────────────────────────────────────────────┐
│ REQUEST PATH WITH REDUNDANT LAYERS │
│ │
│ Client │
│ │ +1-3ms │
│ ▼ │
│ NLB │
│ │ +1-3ms │
│ ▼ │
│ ALB │
│ │ +1-3ms │
│ ▼ │
│ NGINX │
│ │ +1-3ms │
│ ▼ │
│ Envoy sidecar │
│ │ │
│ ▼ │
│ Backend │
│ │
│ Total proxy overhead: 4-12ms │
│ If trace shows 20%+ of p99 latency in proxy hops, │
│ the layers are redundant. │
│ │
│ Fix: assign one job per layer, remove duplicates. │
│ │
└──────────────────────────────────────────────────────────────┘Summary
| Item | Value |
|---|---|
| Reverse proxy | Forward and secure traffic; TLS termination, caching |
| Load balancer | Distribute across replicas; round-robin, least-conn, ip-hash |
| API gateway | Govern API consumers; per-route auth, quotas, analytics |
| NGINX upstream | Defines backend group for load balancing |
| Health checks (OSS) | max_fails, fail_timeout |
| Health checks (Plus) | health_check, active monitoring |
| Session persistence | ip_hash or shared session store |
| Layer anti-pattern | Stacking proxies without distinct responsibilities |
| Gateway latency cost | ~5-9ms p99 versus plain proxy |
| Canonical edge | L4 for distribution, gateway for policy, mesh for mTLS |
Key takeaways:
- The three components share a data path but not a scope. A reverse proxy forwards traffic. A load balancer distributes it. An API gateway governs it with consumer identity and policy . The same tool can do parts of all three, but the responsibility determines which configuration is appropriate.
- NGINX performs both reverse proxy and load balancer roles. A
serverblock withproxy_passis a reverse proxy. Adding anupstreamblock with multiple servers makes it a load balancer . The distinction is configuration, not a different binary. - The load balancing algorithm must match the workload. Round-robin for uniform requests. Least-connections for variable duration. IP hash for session persistence without a shared store . Choosing the wrong one produces imbalance or lost sessions.
- An API gateway is justified by consumer policy, not by proxying. If you have one service and need TLS termination, a reverse proxy costs less and adds less latency. The gateway’s value is per-consumer authentication, quotas, and usage attribution across many services .
- Redundant layers add latency without adding safety. Every proxy hop adds 1 to 3 milliseconds. If a distributed trace shows proxy hops consuming more than 20 percent of end-to-end latency, the architecture has layers that duplicate each other’s jobs .
- Layer separation is a discipline, not a default. The canonical edge assigns L4 for distribution, the gateway for policy, and the mesh for east-west mTLS. A component that does not own a distinct job is a component that should be removed.
Remember: The question is never “which of these three is best.” It is “what job does this layer own, and does another layer already do it?” A reverse proxy owns the boundary between public and private. A load balancer owns distribution across identical replicas. An API gateway owns the policy contract with API consumers. When those responsibilities are clear, the architecture is simple: one layer for each job, no layer for jobs that are already covered. When they are unclear, every new component looks like a solution and becomes a source of latency and operational burden. The discipline of assigning one concern per layer is what separates a scalable edge from a stack of proxies that each do a little of everything and none of it completely.
Stop using slow, ad-bloated tool sites! 🤮
🔎 Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)
⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free
🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!