Docker 90 🐳 Networking Deep Dive: Packet Flow Tracking through iptables, NAT, and veth Pairs
A packet that leaves a container and reaches the internet passes through at least four distinct processing stages, and each stage is implemented by a different kernel subsystem. The veth pair is the virtual cable between the container’s network namespace and the host’s. The Linux bridge is the switch that connects the host-side veth to the host’s other interfaces. The iptables nat table is the address translator that rewrites the source address on the way out and the destination address on the way back. The filter table is the firewall that decides whether the packet is allowed to proceed. The FORWARD chain is where Docker’s rules live, and the DOCKER-USER chain is the first rule in that chain, which is why it is the only chain where user rules take precedence over Docker’s own.
The tracing problem is that the packet is visible at each stage in a different form: the veth interface sees the packet with the container’s IP, the bridge sees it with the same IP, the POSTROUTING chain sees it after the source NAT has been applied, and the destination sees it with the host’s public IP. The conntrack table is the state that ties the outbound packet to the return packet, and the MASQUERADE rule is the mechanism that rewrites the source address to the host’s address on the outbound interface. The DOCKER chain in the nat table handles the DNAT for published ports, and the DOCKER-ISOLATION-STAGE-1 chain prevents one bridge network from reaching another.
This chapter traces a packet from a container to the internet and from the internet to a published port, showing the exact iptables chain and the exact transformation at each step. It covers the veth pair and bridge mechanics, the nat table’s PREROUTING, POSTROUTING, and OUTPUT chains, the filter table’s FORWARD and DOCKER-USER chains, the conntrack state that makes the return path work, and the diagnostic commands that show the rules and the counters. It assumes familiarity with Docker networking basics and the Linux command line.
Key point: Outbound traffic is SNATed in POSTROUTING to the host’s IP. Inbound traffic to a published port is DNATed in PREROUTING to the container’s IP. Both directions are permitted by FORWARD rules that Docker installs. The conntrack table maps the return packet to the original connection, and the DOCKER-USER chain is the first place user rules can override Docker’s defaults.
Why the packet path is hard to trace
The packet changes form at each stage. The container’s network namespace sees the packet with the container’s IP as the source. The host’s bridge sees the same packet with the same source, because the veth pair is a layer-2 cable and no address translation happens at the bridge. The nat table’s POSTROUTING chain rewrites the source to the host’s IP before the packet leaves the host. The return packet arrives with the host’s IP as the destination, and conntrack reverses the translation before the packet is delivered to the container. A trace that only looks at one stage sees one version of the packet and misses the transformations .
The iptables chains are ordered, and the order determines the outcome. For an outbound packet, the path is OUTPUT (for locally generated traffic) or PREROUTING (for forwarded traffic), then FORWARD, then POSTROUTING. For an inbound packet, the path is PREROUTING, then FORWARD, then POSTROUTING. The nat table is consulted in PREROUTING and POSTROUTING, and the filter table is consulted in INPUT, FORWARD, and OUTPUT. A rule in the wrong chain never sees the packet .
Docker’s rules are numerous and interleaved. The DOCKER chain in the nat table handles port publishing. The DOCKER-ISOLATION-STAGE-1 and DOCKER-ISOLATION-STAGE-2 chains prevent bridge networks from reaching each other. The DOCKER-USER chain is the first rule in FORWARD and is the only chain where user rules take precedence over Docker’s. The DOCKER chain in the filter table handles the per-network forwarding rules. The docker-proxy process handles the userland proxy for published ports when userland-proxy is enabled .
The conntrack table is the state that makes it work. A MASQUERADE rule rewrites the source address, but the return packet’s destination is the host’s address, not the container’s. The conntrack table stores the original tuple and the translated tuple, and the nat table’s POSTROUTING chain applies the reverse translation to the return packet before it is delivered to the container. Without conntrack, the return packet would be delivered to the host, not to the container .
The trade-off. The layered design isolates containers and allows many containers to share a single host IP. The cost is that the packet path is not obvious, and debugging a connectivity problem requires tracing the packet through each stage. The iptables -t nat -L -n -v and iptables -t filter -L -n -v commands show the rules and their counters, and conntrack -L shows the active connections. The tcpdump on the veth, the bridge, and the host’s external interface shows the packet at each stage with its current address .
a. The veth pair and the bridge: layer 2
When Docker creates a container on a bridge network, it creates a veth pair: one end in the container’s network namespace, the other end in the host’s network namespace. The container’s end is named eth0 (or the next available name) and has the container’s IP address. The host’s end is named veth<id> and is attached to the bridge (docker0 for the default bridge, or br-<network-id> for a user-defined bridge) .
# List the veth pairs on the host.
ip link show type veth
# Show the bridge and its attached interfaces.
ip link show docker0
bridge link show
# Show the container's interface and address.
docker exec mycontainer ip addr show eth0
The veth pair is a layer-2 cable. A packet that the container sends to its default gateway (the bridge’s IP) is transmitted through the veth to the host’s end, and the bridge forwards it to the appropriate port. The bridge learns MAC addresses, so it forwards the packet to the host’s external interface if the destination is outside the bridge’s subnet. The packet’s IP addresses are unchanged at this stage; only the Ethernet frame is rewritten with the bridge’s MAC address .
The bridge’s IP is the container’s default gateway. For the default bridge, the gateway is 172.17.0.1; for a user-defined bridge, it is the first address in the subnet. The container’s route table points the default route to the bridge’s IP, and the bridge forwards the packet to the host’s routing table, which decides the external interface based on the destination IP .
b. The nat table: SNAT on the way out, DNAT on the way in
The nat table is consulted in PREROUTING (before routing), OUTPUT (locally generated traffic), and POSTROUTING (after routing, before transmission). The MASQUERADE rule in POSTROUTING is the source NAT for outbound traffic: it rewrites the source address to the address of the outbound interface, so the container can reach the internet through the host’s IP .
# Show the nat table rules with counters.
iptables -t nat -L -n -v
# The MASQUERADE rule for the bridge subnet.
# MASQUERADE all -- 172.17.0.0/16 !172.17.0.0/16 -> 0.0.0.0/0
The DOCKER chain in the nat table is the destination NAT for published ports. When a container publishes a port with -p 8080:80, Docker installs a DNAT rule in PREROUTING that rewrites the destination address and port from the host’s address and port to the container’s address and port. The DOCKER chain is the target of the PREROUTING rule for traffic destined for the host’s published ports .
# The DNAT rule for a published port.
# DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 to:172.17.0.2:80
The OUTPUT chain in the nat table handles traffic that originates on the host and is destined for a published port. Without the OUTPUT rule, a process on the host that connects to localhost:8080 would not reach the container, because the packet is locally generated and does not traverse PREROUTING .
c. The filter table and the DOCKER-USER chain
The filter table is consulted in INPUT, FORWARD, and OUTPUT. Docker’s rules are in the FORWARD chain, because the packet from the container is forwarded, not locally delivered. The DOCKER-USER chain is the first rule in FORWARD, and it is the chain where user rules take precedence over Docker’s .
# The FORWARD chain.
iptables -L FORWARD -n -v
# The DOCKER-USER chain.
iptables -L DOCKER-USER -n -v
The DOCKER-USER chain is empty by default. A rule added to it is evaluated before Docker’s own forwarding rules. This is the mechanism for blocking a specific container from reaching the internet, allowing only specific source IPs to reach a published port, or dropping traffic between networks. The rule must be added with -I DOCKER-USER (insert at the top) rather than -A (append), because the chain’s evaluation order is top to bottom .
# Block a specific container subnet from reaching the internet.
iptables -I DOCKER-USER -s 172.18.0.0/16 -o eth0 -j DROP
The DOCKER-ISOLATION-STAGE-1 and DOCKER-ISOLATION-STAGE-2 chains prevent bridge networks from reaching each other. A packet from a container on br-a to a container on br-b is dropped by these chains unless the networks are explicitly connected. The isolation is why two containers on different user-defined networks cannot reach each other by IP, even though they are on the same host .
Complete Example Session
# ============================================
# PART 1: THE VETH PAIR AND BRIDGE
# ============================================
ip link show type veth
# veth1a2b3c4@if2: <BROADCAST,MULTICAST,UP> mtu 1500
# veth5d6e7f8@if3: <BROADCAST,MULTICAST,UP> mtu 1500
ip link show docker0
# docker0: <BROADCAST,MULTICAST,UP> mtu 1500
# inet 172.17.0.1/16
bridge link show
# 3: veth1a2b3c4@if2: <BROADCAST,MULTICAST,UP> master docker0
docker exec mycontainer ip addr show eth0
# eth0: inet 172.17.0.2/16
# ============================================
# PART 2: OUTBOUND PACKET — CONTAINER TO INTERNET
# ============================================
# Container sends packet to 8.8.8.8.
# Source: 172.17.0.2, Dest: 8.8.8.8
# │
# ▼
# veth pair → docker0 bridge
# │
# ▼
# Host routing: 8.8.8.8 via eth0
# │
# ▼
# POSTROUTING: MASQUERADE
# Source rewritten: 172.17.0.2 → 203.0.113.5 (host IP)
# │
# ▼
# Packet leaves eth0: Source 203.0.113.5, Dest 8.8.8.8
# │
# ▼
# conntrack records the tuple.
# ============================================
# PART 3: RETURN PACKET — INTERNET TO CONTAINER
# ============================================
# Reply from 8.8.8.8 to 203.0.113.5.
# │
# ▼
# PREROUTING: conntrack reverses NAT
# Dest rewritten: 203.0.113.5 → 172.17.0.2
# │
# ▼
# FORWARD: DOCKER-USER → DOCKER-ISOLATION → ACCEPT
# │
# ▼
# docker0 bridge → veth → container eth0
# ============================================
# PART 4: INBOUND — PUBLISHED PORT
# ============================================
# External client connects to 203.0.113.5:8080.
# │
# ▼
# PREROUTING: DOCKER chain DNAT
# Dest rewritten: 203.0.113.5:8080 → 172.17.0.2:80
# │
# ▼
# FORWARD: DOCKER-USER → DOCKER chain ACCEPT
# │
# ▼
# docker0 bridge → veth → container eth0
# ============================================
# PART 5: INSPECTING THE NAT TABLE
# ============================================
iptables -t nat -L -n -v
# Chain PREROUTING
# DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 to:172.17.0.2:80
#
# Chain POSTROUTING
# MASQUERADE all -- 172.17.0.0/16 !172.17.0.0/16
# ============================================
# PART 6: INSPECTING THE FILTER TABLE
# ============================================
iptables -L FORWARD -n -v
# Chain FORWARD
# DOCKER-USER all -- anywhere anywhere
# DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
# ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
# ============================================
# PART 7: ADDING A DOCKER-USER RULE
# ============================================
# Block container subnet from reaching the internet.
iptables -I DOCKER-USER -s 172.18.0.0/16 -o eth0 -j DROP
# ============================================
# PART 8: CONNTRACK
# ============================================
conntrack -L | grep 172.17.0.2
# ============================================
# PART 9: TCPDUMP AT EACH STAGE
# ============================================
# On the container's veth:
tcpdump -i veth1a2b3c4 -n
# Source 172.17.0.2, Dest 8.8.8.8
# On the host's external interface:
tcpdump -i eth0 -n
# Source 203.0.113.5, Dest 8.8.8.8
# ============================================
# PART 10: THE COMPLETE FLOW
# ============================================
# Outbound: veth → bridge → routing → POSTROUTING SNAT → eth0
# Inbound: eth0 → PREROUTING conntrack → FORWARD → bridge → veth
# Published: eth0 → PREROUTING DNAT → FORWARD → bridge → veth
# User rules: DOCKER-USER (first in FORWARD)
Quick Reference
Packet Path Stages
| Stage | Chain | Table | Action |
|---|---|---|---|
| Outbound | POSTROUTING | nat | SNAT (MASQUERADE) |
| Inbound (reply) | PREROUTING | nat | Reverse SNAT (conntrack) |
| Published port | PREROUTING | nat | DNAT to container |
| Forward | FORWARD | filter | Accept or drop |
| User rules | DOCKER-USER | filter | First in FORWARD |
Key Commands
| Command | Purpose |
|---|---|
iptables -t nat -L -n -v | Show NAT rules and counters |
iptables -L FORWARD -n -v | Show forwarding rules |
iptables -L DOCKER-USER -n -v | Show user rules |
conntrack -L | Show active connections |
ip link show type veth | List veth pairs |
bridge link show | Show bridge ports |
tcpdump -i <if> -n | Capture packets on interface |
Docker Chains
| Chain | Table | Purpose |
|---|---|---|
DOCKER | nat | Port publishing (DNAT) |
DOCKER | filter | Per-network forwarding |
DOCKER-USER | filter | User rules (first in FORWARD) |
DOCKER-ISOLATION-STAGE-1/2 | filter | Prevent bridge-to-bridge traffic |
Address Transformations
| Direction | Source | Destination |
|---|---|---|
| Outbound (before SNAT) | Container IP | External IP |
| Outbound (after SNAT) | Host IP | External IP |
| Inbound (before DNAT) | External IP | Host IP:port |
| Inbound (after DNAT) | External IP | Container IP:port |
Best Practices
✅ Do This:
# Use DOCKER-USER for custom firewall rules. // ✅
iptables -I DOCKER-USER -s 172.18.0.0/16 -o eth0 -j DROP
# Trace packets at each stage with tcpdump. // ✅
tcpdump -i veth1a2b3c4 -n # container view
tcpdump -i eth0 -n # host view
# Check conntrack for active connections. // ✅
conntrack -L | grep 172.17.0.2
# Use user-defined bridge networks for isolation. // ✅
docker network create --driver bridge mynet
❌ Don’t Do This:
# Add rules to the DOCKER chain directly. // ❌
# Docker may overwrite them; use DOCKER-USER.
# Append to DOCKER-USER instead of insert. // ❌
# Docker's rules may precede yours; use -I.
# Disable iptables management. // ❌
# `--iptables=false` breaks NAT and port publishing.
Common Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
| Container cannot reach internet | MASQUERADE rule missing | Check POSTROUTING in nat |
| Published port not reachable | DNAT rule missing | Check DOCKER chain in nat |
| Custom rule ignored | Added to DOCKER not DOCKER-USER | Use DOCKER-USER with -I |
| Bridge-to-bridge traffic blocked | Isolation chains | Connect networks explicitly |
| Return packet lost | conntrack disabled or full | Check conntrack -L, table size |
Real-World Examples
1. List Veth Pairs
ip link show type veth
2. Show Bridge
bridge link show
3. NAT Rules
iptables -t nat -L -n -v
4. FORWARD Chain
iptables -L FORWARD -n -v
5. DOCKER-USER Rule
iptables -I DOCKER-USER -s 172.18.0.0/16 -o eth0 -j DROP
6. Conntrack
conntrack -L
7. Tcpdump on Veth
tcpdump -i veth1a2b3c4 -n
8. Tcpdump on eth0
tcpdump -i eth0 -n
9. DNAT Rule
# DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 to:172.17.0.2:80
10. MASQUERADE Rule
# MASQUERADE all -- 172.17.0.0/16 !172.17.0.0/16
Visual
Outbound Packet Path
┌──────────────────────────────────────────────┐
│ CONTAINER │
│ src 172.17.0.2 → dst 8.8.8.8 │
│ │ │
│ ▼ │
│ veth pair (layer 2) │
│ │ │
│ ▼ │
│ docker0 bridge (layer 2 forwarding) │
│ │ │
│ ▼ │
│ Host routing: 8.8.8.8 via eth0 │
│ │ │
│ ▼ │
│ POSTROUTING (nat): MASQUERADE │
│ src 172.17.0.2 → 203.0.113.5 │
│ │ │
│ ▼ │
│ eth0: src 203.0.113.5 → dst 8.8.8.8 │
│ │
│ conntrack records the tuple. │
│ │
└──────────────────────────────────────────────┘
Return Packet Path
┌──────────────────────────────────────────────┐
│ eth0: src 8.8.8.8 → dst 203.0.113.5 │
│ │ │
│ ▼ │
│ PREROUTING (nat): conntrack reverse NAT │
│ dst 203.0.113.5 → 172.17.0.2 │
│ │ │
│ ▼ │
│ FORWARD (filter): │
│ DOCKER-USER → DOCKER-ISOLATION → ACCEPT │
│ │ │
│ ▼ │
│ docker0 bridge → veth → container │
│ │
└──────────────────────────────────────────────┘
Published Port Path
┌──────────────────────────────────────────────┐
│ External client → 203.0.113.5:8080 │
│ │ │
│ ▼ │
│ PREROUTING (nat): DOCKER chain DNAT │
│ dst 203.0.113.5:8080 → 172.17.0.2:80 │
│ │ │
│ ▼ │
│ FORWARD (filter): ACCEPT │
│ │ │
│ ▼ │
│ docker0 bridge → veth → container:80 │
│ │
└──────────────────────────────────────────────┘
iptables Chain Order
┌──────────────────────────────────────────────┐
│ OUTBOUND │
│ OUTPUT → POSTROUTING (nat) │
│ (locally generated) │
│ │
│ FORWARDED │
│ PREROUTING (nat) → FORWARD (filter) │
│ → POSTROUTING (nat) │
│ │
│ INBOUND (local delivery) │
│ PREROUTING (nat) → INPUT (filter) │
│ │
│ DOCKER-USER is first in FORWARD. │
│ User rules take precedence there. │
│ │
└──────────────────────────────────────────────┘
Summary
| Item | Value |
|---|---|
| Veth pair | Layer-2 cable between container and host |
| Bridge | Virtual switch connecting veth to host interfaces |
| Outbound SNAT | MASQUERADE in POSTROUTING (nat) |
| Inbound DNAT | DNAT in PREROUTING (nat) for published ports |
| Return translation | conntrack reverses NAT |
| Forwarding | FORWARD chain (filter) |
| User rules | DOCKER-USER (first in FORWARD) |
| Isolation | DOCKER-ISOLATION-STAGE-1/2 |
| Debugging | iptables -L -n -v, conntrack -L, tcpdump |
Key takeaways:
- The packet changes form at each stage. The container sees the container IP, the bridge sees the container IP,
POSTROUTINGrewrites the source to the host IP, and the return packet is rewritten byconntrackbefore delivery. Tracing requires capturing at each stage . - Outbound traffic is SNATed; inbound traffic to a published port is DNATed. The
MASQUERADErule inPOSTROUTINGrewrites the source for outbound. TheDNATrule inPREROUTINGrewrites the destination for published ports. Theconntracktable ties the return packet to the original connection . - The
DOCKER-USERchain is the only place user rules take precedence. It is the first rule in theFORWARDchain. Rules added with-Iare evaluated before Docker’s own forwarding rules. Rules added to theDOCKERchain directly may be overwritten . - The veth pair and bridge are layer 2. They forward Ethernet frames, not IP packets. The IP addresses are unchanged at the bridge; the routing decision happens on the host, and the NAT happens in
POSTROUTINGorPREROUTING. - The isolation chains prevent bridge-to-bridge traffic. A container on one user-defined network cannot reach a container on another by default. The
DOCKER-ISOLATION-STAGE-1/2chains drop the packets before they reach the destination bridge . - Debugging requires multiple tools.
iptables -L -n -vshows the rules and counters,conntrack -Lshows the active connections,ip link show type vethshows the veth pairs, andtcpdumpon the veth and the host interface shows the packet at each stage with its current address .
Remember: A Docker packet is a sequence of transformations, not a single object. It starts in the container with the container’s IP, crosses the veth pair and the bridge as a layer-2 frame, is routed by the host, is rewritten by POSTROUTING to the host’s IP, and is rewritten back by conntrack on the return path. The DOCKER chain handles published ports with DNAT, the FORWARD chain permits the traffic, and the DOCKER-USER chain is the only place user rules can override Docker’s defaults. The tools are iptables, conntrack, ip link, bridge, and tcpdump. The path is deterministic once you know the chains, and the counters in iptables -L -n -v show which rules are matching.
Stop using slow, ad-bloated tool sites! 🤮
🔎 Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)
⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free
🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!