Docker 63 🐳 Docker Swarm Architecture: Manager Nodes, Worker Nodes, and Raft Consensus Protocol
Docker Swarm is Docker’s native clustering and orchestration solution, built directly into the Docker Engine. Unlike Kubernetes, which is a separate system that you install on top of your infrastructure, Swarm mode is activated with a single command and requires no additional software. This integration is both its greatest strength and its most limiting constraint: you can build a production-capable cluster from nothing but Docker Engine instances, but you accept a narrower feature set than Kubernetes offers.
The architecture of a Swarm cluster rests on two distinct node roles and one consensus algorithm. Manager nodes form the control plane—they maintain cluster state, schedule services, and serve the Swarm API. Worker nodes form the data plane—they receive tasks from managers and run containers. The managers coordinate among themselves using the Raft consensus algorithm, which ensures that every manager stores the same consistent view of the cluster. This consistency is what allows any manager to take over if the leader fails.
Understanding Swarm’s architecture means understanding three things: what managers do, what workers do, and how Raft keeps the managers in agreement. The rest—services, tasks, overlay networks, load balancing—builds on this foundation.
Key point: A Docker Swarm cluster consists of manager nodes that form a Raft consensus group for cluster state, and worker nodes that execute tasks dispatched by managers; the Raft quorum requirement means a cluster of N managers can tolerate the loss of at most (N-1)/2 managers.
Why this architecture exists
The single-host problem. Running containers on a single Docker host is fine for development but unacceptable for production. If that host fails, every container on it becomes unavailable. Swarm solves this by clustering multiple Docker Engines into a single logical unit, so that services can run across many machines and survive individual node failures .
The consistency problem. In a distributed cluster, multiple managers must agree on the cluster state—which services exist, how many replicas each has, which nodes are available. If managers disagree, the cluster behaves unpredictably. Swarm solves this with Raft, a consensus algorithm that requires a majority of managers to agree before any state change is committed .
The separation of concerns. Managers handle orchestration and cluster management. Workers handle execution. This separation means that a manager failure does not stop running containers—the workers continue executing their assigned tasks—but it does stop new scheduling decisions until quorum is restored . The architecture is designed so that the control plane and data plane can fail independently.
The decentralized design principle. Docker Engine does not differentiate between node roles at deployment time. Any node can be a manager, a worker, or both. The specialization happens at runtime through promotion and demotion commands . This means you can build an entire Swarm from a single disk image and assign roles after installation.
The fault tolerance requirement. A production cluster must survive failures. Raft provides fault tolerance through replication and quorum: every state change is replicated to a majority of managers before it is committed. If a manager fails, the remaining managers still have the full state, and one of them can become the new leader .
a. Manager nodes: the control plane
Manager nodes are the brain of the Swarm. They handle all cluster management tasks: maintaining cluster state, scheduling services, and serving the Swarm mode HTTP API endpoints .
Cluster state maintenance. Every manager stores a complete copy of the cluster state. This includes the list of nodes, the definitions of all services, the tasks assigned to each node, and the overlay network configurations. The state is maintained in a Raft log and replicated to all managers .
Service scheduling. When you create a service with a desired number of replicas, the manager decides which worker nodes should run each replica. This scheduling decision is based on node availability, resource constraints, and placement preferences. The scheduler only runs on the leader manager .
API endpoints. Managers serve the Swarm mode HTTP API. When you run docker service create on any manager, it forwards the request to the leader, which processes it and commits the change to the Raft log .
Leader election. Manager nodes elect a single leader to conduct orchestration tasks. The leader is the only manager that schedules tasks and commits new state changes. Other managers are followers that replicate the leader’s decisions. If the leader fails, the remaining managers elect a new leader .
Quorum requirements. Raft requires a majority or quorum of managers to agree before any state change is committed. For N managers, the quorum is (N/2)+1. A three-manager cluster requires two managers to be available for any administrative operation. A five-manager cluster requires three .
Fault tolerance. Raft tolerates up to (N-1)/2 failures. A three-manager swarm tolerates one manager failure. A five-manager swarm tolerates two. A seven-manager swarm tolerates three .
Docker’s recommendation. Docker recommends an odd number of managers—three, five, or seven—to maximize fault tolerance while minimizing coordination overhead. Adding more managers does not increase scalability or performance; in general, the opposite is true .
b. Worker nodes: the data plane
Worker nodes are the muscle of the Swarm. Their sole purpose is to execute containers as directed by managers .
Task execution. A task is the atomic unit of scheduling in Swarm. It carries a Docker container and the commands to run inside it. When a manager assigns a task to a worker, the worker’s agent receives the task definition, pulls the required image, and starts the container .
State reporting. Each worker runs an agent that reports the current state of its assigned tasks back to the managers. This feedback loop is how the manager knows whether the actual state of the cluster matches the desired state. If a task fails, the worker reports the failure, and the manager schedules a replacement .
No Raft participation. Worker nodes do not participate in the Raft distributed state, make scheduling decisions, or serve the Swarm API. They are purely execution environments .
Default behavior. By default, manager nodes are also worker nodes. In a single-node Swarm, the one node is both manager and worker, and the scheduler places all tasks on the local engine. To prevent the scheduler from placing tasks on a manager node in a multi-node Swarm, set the manager’s availability to Drain .
Drain mode. When you drain a node, the scheduler gracefully stops tasks on that node and reassigns them to other available workers. The scheduler does not assign new tasks to nodes in Drain mode. This is the standard procedure for taking a node offline for maintenance .
Promotion and demotion. You can promote a worker to manager with docker node promote and demote a manager to worker with docker node demote. Promotion is useful when taking a manager offline for maintenance; demotion is useful when reducing the number of managers .
c. Raft consensus: keeping managers in agreement
Raft is the algorithm that keeps all manager nodes in sync. It ensures that every manager stores the same consistent state, so that any manager can take over if the leader fails .
The problem Raft solves. In a distributed system, multiple nodes must agree on a sequence of values—in Swarm’s case, the sequence of state changes that define the cluster. If managers disagree, the cluster behaves inconsistently. Raft solves this by electing a single leader that serializes all state changes and replicates them to followers .
Leader election. Each manager starts a random election timeout. When a timeout expires, that node requests votes from its peers and becomes a candidate. If it gathers a majority of votes, it becomes the leader. The leader sends periodic heartbeats to followers to maintain its authority. If followers miss heartbeats, they start a new election .
Log replication. When the leader receives a state change request—such as creating a service or scaling replicas—it appends the change as an entry in its Raft log. It then sends the log entry to each follower. Once a majority of followers acknowledge the entry, the change is committed and applied to the cluster state .
Quorum. Raft requires a quorum of (N/2)+1 managers to commit any change. This is why manager counts should be odd: a three-manager cluster has a quorum of two, tolerating one failure. A four-manager cluster also has a quorum of three, but tolerates only one failure—the same fault tolerance as three managers, with more coordination overhead .
Fault tolerance. The formula is (N-1)/2. A five-manager cluster tolerates two simultaneous failures. A seven-manager cluster tolerates three. Docker recommends a maximum of seven managers because beyond that, coordination overhead outweighs any benefit .
Loss of quorum. If a cluster loses quorum—for example, three of five managers go offline—the cluster cannot process any more requests to schedule additional tasks. Existing tasks keep running, but the scheduler cannot rebalance tasks or create new services. The cluster is effectively read-only until quorum is restored .
Recovering from quorum loss. If quorum is lost and the failed managers cannot be restored, you can force a new cluster from the remaining manager with docker swarm init --force-new-cluster. This creates a fresh cluster on the current node while preserving the existing service configurations. Worker nodes remain part of the Swarm .
Complete Example Session
The following session demonstrates initializing a Swarm, inspecting manager and worker roles, and observing quorum behavior.
# ============================================
# PART 1: INITIALIZE THE SWARM
# ============================================
# On the first manager node
docker swarm init --advertise-addr 192.168.1.10
# Output includes:
# Swarm initialized: current node is now a manager
# Worker join command with token
# Manager join command (run 'docker swarm join-token manager')
# ============================================
# PART 2: ADD WORKER NODES
# ============================================
# On each worker node, run the worker join command
docker swarm join --token SWMTKN-1-... 192.168.1.10:2377
# ============================================
# PART 3: ADD MORE MANAGER NODES
# ============================================
# On the first manager, get the manager join token
docker swarm join-token manager
# On each additional manager node
docker swarm join --token SWMTKN-1-... 192.168.1.10:2377
# ============================================
# PART 4: VERIFY CLUSTER STATUS
# ============================================
docker node ls
# Expected output:
# ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
# abc123 manager1 Ready Active Leader
# def456 manager2 Ready Active Reachable
# ghi789 manager3 Ready Active Reachable
# jkl012 worker1 Ready Active
# ============================================
# PART 5: INSPECT MANAGER REACHABILITY
# ============================================
docker node inspect manager1 --format "{{ .ManagerStatus.Reachability }}"
# Output: reachable
docker node inspect manager1 --format "{{ .ManagerStatus.Leader }}"
# Output: true
# ============================================
# PART 6: INSPECT WORKER STATE
# ============================================
docker node inspect worker1 --format "{{ .Status.State }}"
# Output: ready
# ============================================
# PART 7: DEPLOY A SERVICE
# ============================================
docker service create \
--name web \
--replicas 6 \
--publish 8080:80 \
nginx:latest
# The manager schedules 6 tasks across available nodes
# ============================================
# PART 8: OBSERVE TASK DISTRIBUTION
# ============================================
docker service ps web
# Shows each task, its assigned node, and its state
# ============================================
# PART 9: SIMULATE MANAGER FAILURE
# ============================================
# Stop Docker on one manager (not the leader)
# On manager2:
sudo systemctl stop docker
# Check cluster status from a remaining manager
docker node ls
# manager2 shows as "Unreachable" but cluster still has quorum
# ============================================
# PART 10: RESTORE QUORUM
# ============================================
# Restart Docker on manager2
sudo systemctl start docker
# Verify manager2 rejoins
docker node ls
# manager2 shows as "Reachable" againThe ten parts cover initialization, adding workers and managers, verifying cluster status, inspecting manager and worker roles, deploying a service, observing task distribution, simulating a manager failure, and restoring quorum. What the session demonstrates is that the manager/worker separation is visible in every operation: managers accept commands and schedule tasks, workers execute them, and Raft keeps the managers in agreement.
Quick Reference
Node Roles
| Role | Responsibilities | Participates in Raft |
|---|---|---|
| Manager | Cluster state, scheduling, API | Yes |
| Worker | Execute tasks, report state | No |
| Manager+Worker | Both roles (default) | Yes |
Quorum and Fault Tolerance
| Managers | Quorum | Tolerated Failures |
|---|---|---|
| 1 | 1 | 0 |
| 3 | 2 | 1 |
| 5 | 3 | 2 |
| 7 | 4 | 3 |
Raft Properties
| Property | Description |
|---|---|
| Leader election | Random timeout, majority vote |
| Log replication | Leader appends, followers acknowledge |
| Quorum | (N/2)+1 managers required |
| Fault tolerance | (N-1)/2 managers tolerated |
Node Management Commands
| Command | Purpose |
|---|---|
docker swarm init | Create a new swarm |
docker swarm join | Join an existing swarm |
docker node ls | List nodes |
docker node inspect | Inspect a node |
docker node promote | Worker → manager |
docker node demote | Manager → worker |
docker node update --availability drain | Stop scheduling on node |
Best Practices
✅ Do This:
docker swarm init --advertise-addr 192.168.1.10 # ✅ specify address
docker node update --availability drain manager1 # ✅ drain before maintenance
docker node promote worker1 # ✅ promote for HA
docker swarm init --force-new-cluster # ✅ recover from quorum loss❌ Don’t Do This:
# Run a single manager in production # ❌ no fault tolerance
# Use an even number of managers # ❌ no benefit over odd
# Copy raft directory between nodes # ❌ node ID conflict
# Add more than seven managers # ❌ overhead without benefitCommon Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
| Quorum lost | Too many managers failed | Restore managers or force new cluster |
| Manager unreachable | Network or daemon failure | Restart daemon or promote worker |
| No task scheduling | No leader elected | Ensure quorum is healthy |
| Manager overloaded | Running tasks on managers | Drain managers in production |
| Node join fails | Wrong token or address | Regenerate join token |
| Split-brain | Even number of managers | Always use odd numbers |
Real-World Examples
1. Three-manager production cluster
docker swarm init --advertise-addr 10.0.1.10
# Add two more managers for quorum2. Drain manager for maintenance
docker node update --availability drain manager2
# Perform maintenance
docker node update --availability active manager23. Promote worker during manager outage
docker node promote worker34. Force new cluster from surviving manager
docker swarm init --force-new-cluster5. Check manager reachability
docker node inspect manager1 --format "{{ .ManagerStatus.Reachability }}"6. Deploy replicated service
docker service create --name web --replicas 6 --publish 80:80 nginx7. Global service on every node
docker service create --mode global --name monitor monitoring-agent8. Scale service down
docker service scale web=39. Remove a node
docker node rm worker110. Backup swarm state
tar czf /backup/swarm.tgz /var/lib/docker/swarmVisual
Swarm Cluster Architecture
┌──────────────────────────────────────────────┐
│ MANAGER NODES (Control Plane) │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │Manager 1│ │Manager 2│ │Manager 3│ │
│ │(Leader) │ │(Follower│ │(Follower│ │
│ │ │ │ │ │ │ │
│ │ Raft │ │ Raft │ │ Raft │ │
│ └────┬────┘ └────┬────┘ └────┬────┘ │
│ │ │ │ │
│ └────────────┼────────────┘ │
│ │ │
│ Raft consensus: quorum = 2 │
│ │ │
└────────────────────┼─────────────────────────┘
│
┌────────────┼────────────┐
│ │ │
▼ ▼ ▼
┌──────────────────────────────────────────────┐
│ WORKER NODES (Data Plane) │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │Worker 1 │ │Worker 2 │ │Worker 3 │ │
│ │ │ │ │ │ │ │
│ │ Task A │ │ Task B │ │ Task C │ │
│ │ Task D │ │ Task E │ │ Task F │ │
│ └─────────┘ └─────────┘ └─────────┘ │
│ │
│ Workers execute tasks, report state │
└──────────────────────────────────────────────┘Raft Consensus Flow
┌──────────────────────────────────────────────┐
│ LEADER ELECTION │
│ │
│ Manager 1: timer expires, requests votes │
│ Manager 2: votes yes │
│ Manager 3: votes yes │
│ Manager 1: majority reached, becomes leader │
│ │ │
│ ▼ │
│ LOG REPLICATION │
│ │
│ Client → Manager 1 (leader): create service │
│ Manager 1: appends to log │
│ Manager 1 → Manager 2: replicate entry │
│ Manager 1 → Manager 3: replicate entry │
│ Manager 2: acknowledges │
│ Manager 1: majority reached, commits │
│ │ │
│ ▼ │
│ STATE APPLIED │
│ All managers now have the new service │
└──────────────────────────────────────────────┘Quorum and Failure Tolerance
┌──────────────────────────────────────────────┐
│ 3 MANAGERS │
│ ┌─────┐ ┌─────┐ ┌─────┐ │
│ │ M1 │ │ M2 │ │ M3 │ │
│ │ ✅ │ │ ✅ │ │ ❌ │ quorum = 2 │
│ └─────┘ └─────┘ └─────┘ tolerated = 1 │
│ │
│ 5 MANAGERS │
│ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ │
│ │ M1 │ │ M2 │ │ M3 │ │ M4 │ │ M5 │ │
│ │ ✅ │ │ ✅ │ │ ✅ │ │ ❌ │ │ ❌ │ │
│ └─────┘ └─────┘ └─────┘ └─────┘ └─────┘ │
│ quorum = 3, tolerated = 2 │
│ │
│ 7 MANAGERS │
│ quorum = 4, tolerated = 3 │
└──────────────────────────────────────────────┘Manager and Worker Responsibilities
┌──────────────────────────────────────────────┐
│ MANAGER NODE │
│ ┌────────────────────────────────────────┐ │
│ │ ✅ Maintain cluster state │ │
│ │ ✅ Schedule services │ │
│ │ ✅ Serve Swarm API │ │
│ │ ✅ Participate in Raft │ │
│ │ ✅ Elect leader │ │
│ └────────────────────────────────────────┘ │
│ │
│ WORKER NODE │
│ ┌────────────────────────────────────────┐ │
│ │ ✅ Execute tasks │ │
│ │ ✅ Run containers │ │
│ │ ✅ Report task state │ │
│ │ ❌ No Raft participation │ │
│ │ ❌ No scheduling decisions │ │
│ └────────────────────────────────────────┘ │
└──────────────────────────────────────────────┘Summary
| Item | Value |
|---|---|
| Swarm | Cluster of Docker Engines |
| Node | Instance of Docker Engine in swarm |
| Manager | Control plane: state, scheduling, API |
| Worker | Data plane: execute tasks |
| Task | Atomic scheduling unit, carries container |
| Raft | Consensus algorithm for manager agreement |
| Quorum | (N/2)+1 managers required |
| Fault tolerance | (N-1)/2 managers tolerated |
| Recommended managers | 3, 5, or 7 |
| Leader | Single manager that schedules |
| Drain | Stop scheduling tasks on node |
Key takeaways:
- Managers form the control plane. They maintain cluster state, schedule services, and serve the Swarm API. Raft keeps them consistent .
- Workers form the data plane. They execute tasks and report state. They do not participate in Raft or make scheduling decisions .
- Raft requires quorum. A majority of managers must agree before any state change is committed. For N managers, quorum is (N/2)+1 .
- Fault tolerance is (N-1)/2. A three-manager swarm tolerates one failure. A five-manager swarm tolerates two .
- Odd numbers are recommended. Three, five, or seven managers maximize fault tolerance while minimizing coordination overhead. Docker recommends a maximum of seven .
- Managers are workers by default. To reserve managers for control-plane duties, drain them with
docker node update --availability drain. - Quorum loss stops scheduling. Existing tasks keep running, but no new services can be created or scaled until quorum is restored .
- Force new cluster to recover. If quorum cannot be restored,
docker swarm init --force-new-clusteron the surviving manager rebuilds the cluster .
Remember: Docker Swarm’s architecture is built on the separation of control plane and data plane. Managers maintain cluster state, schedule tasks, and serve the API, using Raft to keep themselves in agreement. Workers execute tasks and report state. The Raft quorum requirement—a majority of managers—is what makes the cluster fault-tolerant, and the (N-1)/2 fault tolerance formula is what determines how many manager failures you can survive. Understanding this architecture is the foundation for everything else in Swarm: services, tasks, overlay networks, and rolling updates.
Stop using slow, ad-bloated tool sites! 🤮
🔎 Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)
⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free
🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!