REST API 16 🌐 API Rate Limiting and Throttling (Headers & Algorithms)
Rate limiting and throttling are often used interchangeably, but they solve different problems. Rate limiting rejects requests that exceed a defined quota, returning HTTP 429 and forcing the client to wait or back off. Throttling slows requests down or queues them, allowing the work to complete at a controlled pace rather than failing outright . A public API that returns 429 when a client exceeds 100 requests per minute is rate limiting. A background job processor that accepts 1,000 webhook deliveries and forwards them downstream at 50 per second is throttling. The distinction matters because the algorithm you choose and the headers you return depend on which behavior the client should experience.
The HTTP headers that communicate quota state have been standardized through the IETF’s RateLimit header fields draft. The three core fields are RateLimit-Limit (the quota in the current window), RateLimit-Remaining (units left), and RateLimit-Reset (seconds until reset) . These replace the proliferation of X-RateLimit-* variants that different providers implemented with inconsistent semantics. The RateLimit-Policy field optionally describes multiple windows, such as “10 requests per 5 seconds and 80 requests per 60 seconds” .
The algorithm determines how the counter behaves. Token bucket allows bursts up to a capacity while enforcing a sustained refill rate. Leaky bucket smooths output to a constant rate by queuing excess requests. Fixed window is simple but allows double the limit at window boundaries. Sliding window log is precise but memory-intensive; sliding window counter approximates it with two counters . Token bucket is the most widely deployed for API rate limiting because real clients batch requests and retry on timers—bursts are normal traffic, not abuse .
This chapter covers the IETF header fields and their semantics, the four primary algorithms and their trade-offs, the difference between rejecting and delaying, and the client-side behavior that headers enable. It assumes familiarity with HTTP status codes and the concept of a request quota.
Key point: Rate limiting rejects; throttling delays. The algorithm determines whether bursts are allowed (token bucket) or smoothed away (leaky bucket). The headers tell the client when to retry, but they are hints, not guarantees—RateLimit-Remaining is not a service-level agreement .
Why the headers were standardized
The X-RateLimit-* proliferation created confusion. Before the IETF draft, providers used X-RateLimit-Limit, X-Rate-Limit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, and dozens of variants with different semantics. Some returned reset as seconds remaining; others returned a Unix timestamp; others returned an HTTP date . A client integrating with three APIs had to implement three different parsers. The standardized fields use delta-seconds for RateLimit-Reset specifically to avoid clock synchronization issues between client and server .
The fields are hints, not contracts. The IETF draft is explicit: clients must not consider RateLimit-Remaining as a service-level agreement. If the server is under resource saturation, it may artificially lower the returned value or reject the request regardless of the advertised quota . The headers exist to help well-behaved clients avoid being throttled, not to guarantee that a request with remaining quota will be served.
The reset value can cause a thundering herd. If every client is told the quota resets at 18:00:00, every client may retry at exactly that moment. The draft recommends adding jitter to the reset value or using dynamic throttling policies that return higher reset values under load . This is the same problem as Retry-After with a fixed timestamp.
The trade-off. Standardized headers improve client interoperability and reduce documentation burden. But they expose operational capacity information—a malicious client can probe the endpoint to infer traffic patterns or saturate resources based on the advertised limits . Servers should not disclose capacity to untrusted parties, and should consider whether error responses consume quota.
a. The header fields and their semantics
The three core fields are defined in the IETF draft. RateLimit-Limit is a non-negative integer representing the quota in the current window. RateLimit-Remaining is the number of quota units left. RateLimit-Reset is the number of seconds until the quota resets .
HTTP/1.1 200 OK
RateLimit-Limit: 100
RateLimit-Remaining: 42
RateLimit-Reset: 35The RateLimit-Policy field is optional and describes the policy behind the limit. It can express multiple windows: RateLimit-Policy: 10;w=5, 80;w=60 means 10 units per 5 seconds and 80 units per 60 seconds . This allows a server to enforce both a short-term burst limit and a longer-term average without inventing custom headers.
When the client exceeds the quota, the server returns 429 Too Many Requests with the same headers, so the client knows when to retry. Some servers also return Retry-After, which is a separate header with an HTTP-date or delta-seconds value. Retry-After is not part of the RateLimit draft, but it serves a related purpose: it tells the client exactly how long to wait before the next request .
A critical semantic detail: RateLimit-Reset is delta-seconds, not a timestamp. This avoids the clock-skew problem that plagues timestamp-based reset headers. A client that receives RateLimit-Reset: 35 knows to wait 35 seconds from the moment it received the response, regardless of whether its clock is synchronized with the server’s .
b. The algorithms: token bucket, leaky bucket, fixed window, sliding window
Token bucket models the allowance as a bucket of tokens. Tokens refill at a steady rate up to a maximum capacity. Each request consumes one token. A request that finds the bucket empty is rejected. The bucket capacity determines the maximum burst; the refill rate determines the sustained average . A client allowed 10 requests per second with a capacity of 50 can go quiet for five seconds, fire 50 requests at once, and then continue at 10 per second while the bucket refills. This is the most common algorithm for public APIs because it accommodates legitimate bursts without letting the long-run average drift .
Leaky bucket models the allowance as a queue that drains at a constant rate. Requests enter the queue; if the queue is full, new requests are rejected. The output rate is always constant, regardless of input burstiness . The trade-off is latency: requests wait in the queue rather than failing fast. This algorithm suits fragile downstreams—a legacy database, a third-party API with a hard ceiling—where bursts are dangerous even if the long-term average is fine . NGINX’s limit_req module uses the leaky bucket method, with the burst parameter controlling queue size and nodelay allowing immediate processing of queued requests up to the burst limit .
Fixed window divides time into fixed intervals (e.g., one minute) and counts requests per client per window. It is simple and memory-efficient (one counter per client), but it allows up to double the limit at window boundaries. A client can send 100 requests at 11:59:59 and another 100 at 12:00:00, producing 200 requests in two seconds against a 100-per-minute limit .
Sliding window log stores a timestamp for every request and counts how many fall within the last N seconds. It is perfectly accurate—no boundary burst—but memory-intensive. For 1 million users at 100 requests per minute, it stores 100 million timestamps . Sliding window counter approximates the log by keeping two counters (current and previous window) and weighting the previous window’s count by how much it overlaps the rolling window. It uses two integers instead of an entry per request and removes nearly all of the boundary spike, which is why it is the default in many production gateways .
The practical rule: use token bucket for public APIs where bursts are normal. Use leaky bucket for background jobs or fragile downstreams where smooth output matters more than latency. Use sliding window when fair enforcement across time is more important than raw throughput.
c. Client-side behavior: what the headers enable
The headers exist to let clients self-regulate. A client that reads RateLimit-Remaining and sees it approaching zero can slow down before it receives a 429. A client that receives a 429 with RateLimit-Reset: 30 knows to wait 30 seconds before retrying. This is the cooperative model: the server advertises its policy, and the client respects it.
The client-side retry pattern that works: exponential backoff with jitter. The client waits 1 second, then 2, then 4, up to a maximum, adding random jitter to each wait to prevent synchronized retries from all clients hitting the server at the same moment . The client should also honor Retry-After if present, and check RateLimit-Remaining before firing the next request in a batch.
A client that ignores the headers and retries immediately after a 429 turns transient throttling into a sustained outage. The server’s rate limiter rejects the request; the client retries instantly; the limiter rejects again. The load on the server increases because each rejected request consumes CPU before being rejected. This is the failure mode the headers are designed to prevent.
For API providers, the headers are also a documentation substitute. Instead of documenting “you can make 100 requests per minute” in a static page that may drift from the implementation, the headers communicate the actual policy at runtime. The client can adapt without reading documentation .
Complete Example Session
# ============================================
# PART 1: SUCCESSFUL RESPONSE WITH HEADERS
# ============================================
HTTP/1.1 200 OK
Content-Type: application/json
RateLimit-Limit: 100
RateLimit-Remaining: 42
RateLimit-Reset: 35
RateLimit-Policy: 100;w=60
{
"data": "response body"
}
# ============================================
# PART 2: THROTTLED RESPONSE
# ============================================
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 12
Retry-After: 12
{
"error": "rate_limit_exceeded",
"message": "Too many requests. Retry after 12 seconds."
}
# ============================================
# PART 3: MULTI-WINDOW POLICY
# ============================================
RateLimit-Policy: 10;w=5, 80;w=60
# 10 requests per 5 seconds AND 80 per 60 seconds.
# The closest limit to reach is the 5-second window.
# ============================================
# PART 4: TOKEN BUCKET MENTAL MODEL
# ============================================
# Capacity: 50 tokens
# Refill rate: 10 tokens/second
#
# Client idle for 5 seconds → 50 tokens (capacity cap)
# Fires 50 requests instantly → all allowed
# Bucket empty. Next request rejected until refill.
# After 1 second → 10 tokens → 10 requests allowed.
#
# Sustained rate: 10/second. Burst: 50.
# ============================================
# PART 5: LEAKY BUCKET MENTAL MODEL
# ============================================
# Queue capacity: 20
# Drain rate: 5 requests/second
#
# 15 requests arrive instantly → all queued.
# Processed at 5/second → takes 3 seconds.
# 25 requests arrive → 20 queued, 5 rejected.
#
# Output is smooth; latency is added.
# ============================================
# PART 6: FIXED WINDOW BOUNDARY PROBLEM
# ============================================
# Limit: 100 requests per minute.
# Window 1 (12:00:00–12:00:59): 100 requests at 12:00:59.
# Window 2 (12:01:00–12:01:59): 100 requests at 12:01:00.
#
# 200 requests in 2 seconds against 100/min limit.
# Fixed window allows this. Sliding window does not.
# ============================================
# PART 7: CLIENT RETRY WITH BACKOFF
# ============================================
# On 429:
# wait = Retry-After (if present) or RateLimit-Reset
# add jitter: wait += random(0, wait * 0.1)
# retry
# if repeated 429: exponential backoff (1s, 2s, 4s...)
# maximum backoff cap (e.g., 60s)
# ============================================
# PART 8: NGINX LEAKY BUCKET CONFIG
# ============================================
# limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
#
# location /api/ {
# limit_req zone=api burst=20 nodelay;
# }
#
# rate=10r/s → drain rate.
# burst=20 → queue capacity.
# nodelay → process burst immediately, enforce rate over time.
# ============================================
# PART 9: WHAT THE HEADERS DO NOT MEAN
# ============================================
# RateLimit-Remaining: 42 does NOT guarantee
# the next 42 requests will succeed.
# The server may lower the value under load
# or reject regardless of the advertised quota.
# Headers are hints, not SLAs.
# ============================================
# PART 10: THE COMPLETE FLOW
# ============================================
# 1. Server chooses algorithm (token bucket for APIs).
# 2. Server enforces quota per client key.
# 3. Response includes Limit, Remaining, Reset.
# 4. Client reads Remaining, slows down before 429.
# 5. On 429: client waits Reset + jitter, retries.
# 6. Server may adjust values under load.
# Headers communicate; algorithms enforce.Quick Reference
IETF RateLimit Headers
| Header | Meaning | Example |
|---|---|---|
RateLimit-Limit | Quota in current window | 100 |
RateLimit-Remaining | Units left | 42 |
RateLimit-Reset | Seconds until reset | 35 |
RateLimit-Policy | Optional policy description | 100;w=60 |
Retry-After | Separate header, wait time | 12 or HTTP-date |
Algorithm Comparison
| Algorithm | Burst Handling | Memory | Best For |
|---|---|---|---|
| Token bucket | Controlled bursts up to capacity | Low | Public APIs |
| Leaky bucket | No bursts; queued or rejected | Queue size | Fragile downstreams |
| Fixed window | 2× boundary burst | Very low | Simple internal limits |
| Sliding window log | None (exact) | High | Low-volume accuracy |
| Sliding window counter | Minimal (approximate) | Low | API rate limiting at scale |
Rate Limiting vs Throttling
| Aspect | Rate Limiting | Throttling |
|---|---|---|
| Excess requests | Rejected (429) | Delayed or queued |
| Algorithm | Token bucket, fixed window | Leaky bucket (queue) |
| Latency added | No | Yes |
| Use case | Abuse prevention, quotas | Traffic spikes, fragile downstreams |
NGINX Leaky Bucket Parameters
| Parameter | Effect |
|---|---|
rate=10r/s | Drain rate |
burst=20 | Queue capacity |
nodelay | Process burst immediately, enforce rate over time |
Best Practices
✅ Do This:
# Return standard headers on every response. // ✅
RateLimit-Limit: 100
RateLimit-Remaining: 42
RateLimit-Reset: 35
# Use delta-seconds for Reset, not timestamps. // ✅
RateLimit-Reset: 35
# Return 429 with Retry-After on throttle. // ✅
HTTP/1.1 429 Too Many Requests
Retry-After: 12
# Use token bucket for public APIs. // ✅
# Allows legitimate bursts while enforcing average.
# Client: exponential backoff with jitter on 429. // ✅
wait = reset + random_jitter❌ Don’t Do This:
# Use non-standard X-RateLimit headers. // ❌
X-RateLimit-Limit: 100 # Prefer standard RateLimit-Limit.
# Return Reset as a Unix timestamp. // ❌
RateLimit-Reset: 1735689600 # Clock skew; use delta-seconds.
# Treat Remaining as a guarantee. // ❌
# Server may reject despite Remaining > 0 under load.
# Client: retry immediately after 429. // ❌
# Turns transient throttle into sustained outage.Common Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
| Boundary burst with fixed window | 100 requests at 11:59:59, 100 at 12:00:00 | Sliding window or token bucket |
| Thundering herd on reset | All clients retry at the same timestamp | Add jitter to Reset value |
| Client ignores Remaining | Retries until 429 | Read headers, back off proactively |
| Leaky bucket latency | Requests queued, not rejected | Use token bucket if latency matters |
| Headers on cached responses | Cached response has stale Remaining | Ignore RateLimit on cached responses |
Real-World Examples
1. Token Bucket API Limit
RateLimit-Limit: 100
RateLimit-Remaining: 42
RateLimit-Reset: 352. 429 with Retry-After
HTTP/1.1 429 Too Many Requests
Retry-After: 123. Multi-Window Policy
RateLimit-Policy: 10;w=5, 80;w=604. NGINX Leaky Bucket
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
limit_req zone=api burst=20 nodelay;5. Client Backoff
wait = int(response.headers.get('RateLimit-Reset', 60))
wait += random.uniform(0, wait * 0.1) # jitter
time.sleep(wait)6. Token Bucket Capacity
# capacity=50, refill_rate=10/s
# Idle 5s → 50 tokens → 50 burst allowed.7. Leaky Bucket Queue
# queue_size=20, drain_rate=5/s
# 25 arrive → 20 queued, 5 rejected.8. Ignore Cached Headers
if response.headers.get('Age', 0) > 0:
# Stale RateLimit values; ignore.
pass9. 429 Error Body
{"error": "rate_limit_exceeded", "message": "Retry after 12 seconds"}10. Jitter on Reset
# Server adds random 0-3s to Reset value.
# Prevents synchronized retry storm.Visual
Token Bucket
┌──────────────────────────────────────────────┐
│ TOKEN BUCKET │
│ │
│ Capacity: 50 tokens │
│ Refill: 10 tokens/second │
│ │
│ Request → consume 1 token │
│ │ │
│ ├── Token available → allow │
│ └── No token → reject (429) │
│ │
│ Idle → tokens accumulate to capacity │
│ Burst → spend accumulated tokens │
│ Sustained → refill rate limits average │
│ │
│ Bursts allowed, average enforced. │
│ │
└──────────────────────────────────────────────┘Leaky Bucket
┌──────────────────────────────────────────────┐
│ LEAKY BUCKET │
│ │
│ Requests → Queue (capacity N) │
│ │ │
│ ▼ │
│ Drain at constant rate │
│ │ │
│ ▼ │
│ Downstream (smooth output) │
│ │
│ Queue full → reject │
│ Queue not full → delay, then process │
│ │
│ No bursts passed through. │
│ Latency added. │
│ │
└──────────────────────────────────────────────┘Fixed Window Boundary Problem
┌──────────────────────────────────────────────┐
│ FIXED WINDOW │
│ │
│ Window 1 Window 2 │
│ 12:00:00 12:01:00 │
│ │ │ │
│ 100 requests 100 requests │
│ at 12:00:59 at 12:01:00 │
│ │ │ │
│ └────────┬────────┘ │
│ ▼ │
│ 200 requests in 2 seconds │
│ Limit: 100/minute │
│ Boundary burst allows 2× limit. │
│ │
│ Sliding window prevents this. │
│ │
└──────────────────────────────────────────────┘Rate Limiting vs Throttling
┌──────────────────────────────────────────────┐
│ RATE LIMITING THROTTLING │
│ │
│ Reject (429) Delay/queue │
│ │
│ Token bucket Leaky bucket │
│ Fixed window (queue mode) │
│ │
│ No latency Latency added │
│ │
│ Abuse prevention Traffic shaping │
│ Quotas Fragile downstreams │
│ │
│ Client retries Client waits │
│ with backoff for response │
│ │
│ Choose based on whether the client │
│ can tolerate waiting or needs fast failure. │
│ │
└──────────────────────────────────────────────┘Summary
| Item | Value |
|---|---|
RateLimit-Limit | Quota in current window |
RateLimit-Remaining | Units left (hint, not SLA) |
RateLimit-Reset | Seconds until reset (delta-seconds) |
RateLimit-Policy | Optional multi-window policy |
| Token bucket | Bursts allowed up to capacity |
| Leaky bucket | Smooth output, no bursts |
| Fixed window | Simple, boundary burst problem |
| Sliding window counter | Approximate, low memory |
| Rate limiting | Reject with 429 |
| Throttling | Delay or queue |
Key takeaways:
- The IETF headers replace
X-RateLimit-*with consistent semantics.RateLimit-Limit,RateLimit-Remaining, andRateLimit-Resetcommunicate quota state, andRateLimit-Resetuses delta-seconds to avoid clock skew . RateLimit-Remainingis a hint, not a guarantee. The server may lower the value or reject requests under load. Clients must not treat it as a service-level agreement .- Token bucket allows controlled bursts; leaky bucket smooths output. Token bucket is the default for public APIs because real clients batch requests. Leaky bucket suits fragile downstreams where bursts are dangerous .
- Fixed window allows 2× the limit at boundaries. A client can send the full quota at the end of one window and the full quota at the start of the next. Sliding window counter eliminates most of this at low memory cost .
- Rate limiting rejects; throttling delays. The distinction determines whether the client sees a 429 and retries with backoff, or waits for a queued request to complete .
- NGINX uses leaky bucket.
limit_req_zonedefines the rate,limit_reqapplies it, andburstwithnodelayallows controlled bursts without artificial delay . - Client-side backoff with jitter is essential. Retrying immediately after a 429 turns transient throttling into a sustained outage. Exponential backoff with random jitter, honoring
Retry-AfterandRateLimit-Reset, is the correct pattern .
Remember: Rate limiting and throttling are the same mechanism viewed from different ends. The algorithm determines whether excess requests are rejected or delayed. The headers communicate the policy so well-behaved clients can self-regulate before they hit the limit. Token bucket is the default because it matches how real clients behave—bursty but bounded. Leaky bucket is for the cases where the downstream cannot tolerate any burst, even a legitimate one. The headers are hints; the algorithm is the enforcement. A client that ignores the hints will be rejected, and a client that retries without backoff will make the rejection worse.
Stop using slow, ad-bloated tool sites! 🤮
🔎 Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)
⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free
🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!