Rust 118 🦀 Custom Async Executors and Polling Reactor/Waker Architecture - KandZ

⏱️ Estimated read: 13 min | |

Rust’s standard library provides the Future trait, the Waker type, and the Context that ties them together. It does not provide an executor. Every async runtime in the ecosystem—Tokio, async-std, smol, and every educational implementation—builds these missing pieces from the same primitives. Understanding this architecture is not academic. When a future hangs, when a task never wakes, when a reactor misses an event, the debugging path runs directly through the executor-reactor boundary and the Waker contract that connects them.

Key point: The executor polls futures; the reactor listens for external events; the Waker is the bridge. A future returns Poll::Pending, hands its Waker to a reactor, and the reactor calls wake() when the event arrives. Without that cycle, the future is suspended forever.

Why custom executors and reactors exist

The standard library gap. std::future::Future defines an interface, not an engine. A Future is an inert state machine: calling poll advances it if it can make progress, but nothing calls poll unless an executor does. The std library stops at Waker and Context; the scheduling loop, the task queue, and the event notification system are left to the ecosystem .

The monolithic runtime problem. Tokio is excellent, but it is large. A program that needs async I/O and a work-stealing scheduler does not need the full Tokio surface. Educational runtimes like yot-run demonstrate that the core components—executor, reactor, task, waker—can be implemented in a few thousand lines without pulling in the entire ecosystem . Custom executors exist because sometimes the right runtime is the one you understand completely.

The reactor/executor separation. The async book defines a runtime as a bundle: a reactor plus one or more executors . The executor owns the task queue and drives futures. The reactor owns OS file descriptors, timers, and other external event sources. Leaf futures—TcpStream::read, Sleep—register their Waker with the reactor. When data arrives or the timer expires, the reactor calls wake(), which puts the task back on the executor’s queue. This separation is the architectural spine of every async runtime.

The Waker contract. A Waker is a handle that, when woken, notifies the executor that the associated task is ready to be polled again. The RawWakerVTable defines four functions: clone, wake, wake_by_ref, and drop. The executor constructs a RawWaker from its task representation—usually an Arc<Task>—and passes it inside a Context when polling. The future stores this Waker somewhere (typically with the reactor) and calls wake() later . If the Waker is cloned, the task is incremented; if dropped, decremented; if woken, enqueued. Every reference count and memory safety invariant in the async runtime flows through these four function pointers.

The trade-off. Building a custom executor means owning the subtle bugs: lost wakeups, double wakes, tasks polled after completion. Using an existing runtime means accepting its footprint and its ecosystem compatibility constraints. The choice depends on whether the learning exercise or the production requirement dominates.

a. The executor loop and task representation

An executor is a loop that pops tasks from a queue and polls them. A task is a pinned future plus the state needed to wake it. In the simplest single-threaded executor, the task is Pin<Box<dyn Future<Output = ()>>>, and the queue is a VecDeque<Rc<Task>>.

type BoxedFuture = Pin<Box<dyn Future<Output = ()>>>;

struct Task(RefCell<BoxedFuture>);

impl LocalWake for Task {
    fn wake(self: Rc<Self>) {
        RUN_QUEUE.with_borrow_mut(|queue| {
            queue.push_back(self);
        });
    }
}

The LocalWake trait from the alloc crate provides the building block: it constructs a LocalWaker from an Rc, avoiding the atomic overhead of Arc . The wake implementation pushes the task back onto the run queue. When the executor pops it, it polls the future again. This is the entire scheduling cycle: poll, pend, wake, requeue, poll.

The block_on function drives the loop. It spawns the root future, then repeatedly pops tasks and polls them until the queue is empty.

fn block_on<F: Future<Output = ()> + 'static>(future: F) {
    spawn(future);
    loop {
        let Some(task) = RUN_QUEUE.with_borrow_mut(|q| q.pop_front()) else {
            return;
        };
        let local_waker: LocalWaker = task.clone().into();
        let mut cx = ContextBuilder::from_waker(Waker::noop())
            .local_waker(&local_waker)
            .build();
        let _ = task.0.borrow_mut().as_mut().poll(&mut cx);
    }
}

The Waker::noop() fallback is a temporary placeholder. The real Waker is derived from the task itself via From<Rc<Task>> for LocalWaker. The future stores this LocalWaker; when the reactor fires, it calls wake(), which pushes the task back onto RUN_QUEUE .

b. The reactor and event registration

The reactor waits on external events. On Linux, that means epoll; on macOS, kqueue. The architecture is the same across platforms: the reactor owns an OS event loop handle, futures register their interest by submitting a Waker and a token, and the reactor calls wake() on matching tokens when the OS signals readiness.

// Simplified reactor interface
impl Reactor {
    pub fn register(&self, fd: RawFd, waker: Waker) -> io::Result<Token>;
    pub fn deregister(&self, token: Token) -> io::Result<()>;
    pub fn poll(&self, timeout: Option<Duration>) -> io::Result<()>;
}

A TcpStream::read future does not block. It checks whether the socket is readable. If not, it calls reactor.register(fd, cx.waker().clone()) and returns Poll::Pending. The reactor stores the mapping from fd to Waker. When epoll_wait returns indicating the socket has data, the reactor looks up the token, calls wake() on the stored Waker, and the executor polls the future again. This time, read succeeds and returns Poll::Ready.

The applied-cs-projects async runtime implements this with an edge-triggered epoll reactor that maps Tokens to Wakers . The yot-run crate documents the same separation: the reactor runs in a background thread, monitoring OS I/O events and waking tasks when data is ready . The reactor is the only component that blocks. The executor never blocks; it only polls tasks that are ready.

c. The `Waker` vtable and manual construction

For multi-threaded executors, the Waker must be Send + Sync, and the task representation uses Arc<Task>. The RawWakerVTable is constructed manually with function pointers for clone, wake, wake_by_ref, and drop.

unsafe fn clone_waker<T: Wake + Send + Sync + 'static>(waker: *const ()) -> RawWaker {
    Arc::increment_strong_count(waker as *const T);
    RawWaker::new(
        waker,
        &RawWakerVTable::new(clone_waker::<T>, wake::<T>, wake_by_ref::<T>, drop_waker::<T>),
    )
}

unsafe fn wake<T: Wake + Send + Sync + 'static>(waker: *const ()) {
    let waker = Arc::from_raw(waker as *const T);
    <T as Wake>::wake(waker);
}

unsafe fn drop_waker<T: Wake + Send + Sync + 'static>(waker: *const ()) {
    Arc::decrement_strong_count(waker as *const T);
}

The std library provides impl From<Arc<W>> for Waker where W: Wake + Send + Sync . The Wake trait is the ergonomic surface; the RawWakerVTable is the machinery underneath. When you implement Wake for your task type and call Arc::new(task).into(), the From impl constructs the vtable with the correct function pointers.

The manual vtable exists because the Waker must be type-erased. The executor passes a Waker to the future; the future does not know the concrete task type. The vtable is the dynamic dispatch mechanism: wake() on the Waker calls wake::<Task>() through the function pointer, reconstructing the Arc<Task> from the raw pointer and invoking the task’s wake implementation. This is the same pattern the standard library uses internally .

A subtle but critical detail: the clone function must increment the reference count atomically. If two threads clone the same Waker concurrently, the Arc strong count must be updated correctly. The vtable functions are the only place where this atomicity is enforced. A bug here—a missing increment, a decrement on the wrong pointer—causes use-after-free or double-free, both of which are undefined behavior.

Complete Example Session

# ============================================
# PART 1: THE TASK TYPE
# ============================================

// A task wraps a pinned future.
type BoxedFuture = Pin<Box<dyn Future<Output = ()>>>;
struct Task {
    future: RefCell<BoxedFuture>,
}

# ============================================
# PART 2: THE WAKE IMPLEMENTATION
# ============================================

use std::task::Wake;
use std::sync::Arc;

impl Wake for Task {
    fn wake(self: Arc<Self>) {
        // Re-enqueue the task on the executor's queue.
        SCHEDULER.enqueue(self);
    }
}

# ============================================
# PART 3: THE EXECUTOR LOOP
# ============================================

loop {
    let task = SCHEDULER.pop();
    let waker = Arc::clone(&task).into();
    let mut cx = Context::from_waker(&waker);
    let _ = task.future.borrow_mut().as_mut().poll(&mut cx);
}

# ============================================
# PART 4: THE REACTOR REGISTRATION
# ============================================

impl Reactor {
    fn register(&self, fd: RawFd, waker: Waker) {
        // Map fd to waker in the reactor's internal table.
        self.wakers.insert(fd, waker);
    }
}

# ============================================
# PART 5: THE REACTOR EVENT LOOP
# ============================================

fn reactor_loop(reactor: Arc<Reactor>) {
    loop {
        let events = epoll_wait(reactor.epoll_fd, ...);
        for event in events {
            if let Some(waker) = reactor.wakers.remove(&event.fd) {
                waker.wake();
            }
        }
    }
}

# ============================================
# PART 6: A SIMPLE TIMER FUTURE
# ============================================

struct Sleep {
    deadline: Instant,
    waker: Option<Waker>,
}

impl Future for Sleep {
    type Output = ();
    fn poll(mut self: Pin<&mut Self>, cx: &mut Context) -> Poll<()> {
        if Instant::now() >= self.deadline {
            return Poll::Ready(());
        }
        self.waker = Some(cx.waker().clone());
        REACTOR.register_timer(self.deadline, cx.waker().clone());
        Poll::Pending
    }
}

# ============================================
# PART 7: THE RAW WAKER VTABLE
# ============================================

unsafe fn wake_task(ptr: *const ()) {
    let task = Arc::from_raw(ptr as *const Task);
    task.wake();
}

unsafe fn clone_task(ptr: *const ()) -> RawWaker {
    Arc::increment_strong_count(ptr as *const Task);
    RawWaker::new(ptr, &VTABLE)
}

unsafe fn drop_task(ptr: *const ()) {
    Arc::decrement_strong_count(ptr as *const Task);
}

# ============================================
# PART 8: MULTI-THREADED WORK STEALING
# ============================================

// The scheduler holds a global injector queue.
// Each worker has a local queue and steals from others.
struct Scheduler {
    injector: Injector<Arc<Task>>,
    workers: Vec<Worker>,
}

# ============================================
# PART 9: THE COMPLETE RUNTIME
# ============================================

// Runtime spawns the reactor thread and executor threads.
// Tasks flow from injector → worker → poll.
// Reactor wakes tasks by pushing back to injector.

The nine parts trace the architecture from task representation through the wake implementation, the executor loop, the reactor registration, the event loop, a timer future, the raw vtable, the work-stealing scheduler, and the complete runtime. Every runtime, from yot-run to Tokio, builds on these same primitives.

Quick Reference

Core Components

Component	Responsibility	Key Type
Executor	Polls tasks, manages queue	`VecDeque<Runnable>` or work-stealing deque
Reactor	Waits on OS events, wakes tasks	`epoll`/`kqueue` + `HashMap<Token, Waker>`
Task	Pinned future + wake logic	`Arc<Task>`
Waker	Handle to wake a specific task	`RawWaker` + `RawWakerVTable`

Waker VTable Functions

Function	Purpose	Reference Count
`clone`	Duplicate the waker	Increment
`wake`	Consume waker, schedule task	Consume
`wake_by_ref`	Schedule without consuming	No change
`drop`	Release waker resources	Decrement

Executor Types

Type	Queue	Task Bound
Single-threaded	`VecDeque<Rc<Task>>`	`!Send`
Multi-threaded	Work-stealing deque	`Send + Sync`

Best Practices

✅ Do This:

impl Wake for Task {
    fn wake(self: Arc<Self>) {
        SCHEDULER.enqueue(self);
    }
}

✅ Wake trait constructs the vtable automatically

let waker = Arc::clone(&task).into();
let mut cx = Context::from_waker(&waker);

✅ Clone the Arc before constructing the Context

REACTOR.register(fd, cx.waker().clone());
Poll::Pending

✅ Register the waker before returning Pending

❌ Don’t Do This:

REACTOR.register(fd, cx.waker().clone());
// forgot to return Pending
Poll::Ready(())

❌ The future must return Pending after registering

unsafe fn wake(ptr: *const ()) {
    let task = Arc::from_raw(ptr as *const Task);
    // forgot to call wake on task
}

❌ from_raw without a corresponding action leaks the Arc

// two futures share the same waker
// wake() called twice

❌ A waker can be woken multiple times; the executor must handle spurious wakes

Common Pitfalls

Pitfall	Why It Happens	Fix
Task never woken	Reactor registration omitted or waker dropped	Store waker before `Pending`
Double poll after completion	Waker called after `Ready`	Executor checks completion state
Use-after-free in vtable	Incorrect reference count	Atomic increment/decrement
Lost wakeup	Waker overwritten before reactor fires	Use `Option<Waker>` and replace
Busy loop	Reactor returns immediately without events	Block with timeout in reactor loop

Real-World Examples

1. Single-Threaded Executor

RUN_QUEUE.with_borrow_mut(|q| q.push_back(task));

2. Multi-Threaded Scheduler

self.injector.push(task);

3. Waker from Arc

let waker: Waker = Arc::new(task).into();

4. Reactor Registration

reactor.register(fd, cx.waker().clone());

5. Timer Wake

timers.insert(deadline, waker);

6. Manual VTable

RawWakerVTable::new(clone, wake, wake_by_ref, drop)

7. Block On

let mut cx = Context::from_waker(&waker);
future.poll(&mut cx)

8. Work Stealing

worker.local.pop().or_else(|| steal())

9. Spurious Wake Check

if woken.swap(false, Ordering::Acquire) { poll() }

10. Epoll Wait

epoll_wait(epoll_fd, &mut events, timeout)

Visual

Executor-Reactor Cycle

┌──────────────────────────────────────────────┐
│  Executor loop                               │
│                                              │
│  pop task → poll future                      │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Future::poll(cx)                            │
│                                              │
│  Can make progress?                          │
│  ├─ YES → Poll::Ready, done                  │
│  └─ NO  → register cx.waker() with reactor   │
│           return Poll::Pending               │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Reactor waits on OS events                  │
│                                              │
│  epoll_wait returns → lookup waker           │
│  → waker.wake() → task back on queue         │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Executor pops task again                    │
│  Polls future → now Ready                    │
└──────────────────────────────────────────────┘

The cycle repeats until the future returns Ready. Every suspension point is a Pending with a registered waker.

Waker Reference Count

┌──────────────────────────────────────────────┐
│  Arc<Task> (strong = 1)                      │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Waker created (strong = 2)                  │
│  (executor holds one, waker holds one)       │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Future stores waker.clone() (strong = 3)    │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Reactor stores waker (strong = 4)           │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Reactor fires: waker.wake()                 │
│  (strong = 3, waker consumed)                │
└──────────────────────────────────────────────┘

Every clone increments; every drop decrements. The task lives as long as any waker or the executor holds a reference.

Summary

Item	Value
Executor role	Polls tasks, manages queue
Reactor role	Waits on OS events, wakes tasks
Task representation	`Arc<Task>` wrapping `Pin<Box<dyn Future>>`
Waker construction	`RawWakerVTable` with clone/wake/drop
Ergonomic trait	`Wake` for `Arc<W>` → `Waker` conversion
Single-threaded waker	`LocalWake` with `Rc`
Registration	Store waker in reactor before `Pending`
Wake behavior	Push task back onto executor queue

Key takeaways:

The executor and reactor are separate. The executor polls; the reactor waits. The Waker bridges them. Without registration, a pending future is never woken .
A task is a pinned future plus wake state. The Arc<Task> is the concrete type behind the type-erased Waker. Every clone and drop manipulates the reference count .
The Wake trait simplifies vtable construction. Implementing Wake for Arc<T> and converting via .into() produces a correct RawWakerVTable automatically .
LocalWake avoids atomic overhead. For single-threaded executors, Rc<T> and LocalWaker eliminate the need for Send + Sync and atomic reference counting .
The reactor blocks; the executor never does. epoll_wait or kqueue waits for events. The executor loop runs continuously, polling ready tasks only.
Spurious wakes are allowed and must be handled. A waker may be called multiple times or when the task is not actually ready. The executor polls; the future returns Pending again if necessary.

Remember: A custom async executor is not a mystery. It is a queue, a loop, and a vtable. The executor pops tasks and polls them. The reactor waits on file descriptors and calls wake(). The Waker is a reference-counted handle that connects the two, ensuring that a future suspended on I/O is resumed when the I/O completes. Every async runtime, from the smallest educational project to Tokio, implements this same cycle. The differences are in the scheduler (work-stealing vs single-threaded), the reactor (epoll vs kqueue vs IOCP), and the task management details. The architecture is universal.

Was this helpful?

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!