Rust 127 🦀 High-Frequency Trading (HFT) Memory Management and Latency Optimization - KandZ

⏱️ Estimated read: 11 min | |

In high-frequency trading, latency is measured in nanoseconds, and memory allocation is the enemy. A single call to the system allocator can cost hundreds of nanoseconds—an eternity when competing orders are separated by microseconds. Rust’s ownership model and zero-cost abstractions make it a viable language for HFT, but the default allocator and standard data structures are not designed for the hot path. Optimization is the discipline of removing every non-deterministic operation from the critical loop: no heap allocations, no lock contention, no unnecessary system calls, and no cache misses.

Key point: The hot path must be allocation-free and lock-free. Memory is pre-allocated before trading begins, inter-thread communication uses cache-aligned ring buffers, and every structure is laid out to maximize cache locality.

Why HFT memory management is different

The allocator problem. The system allocator (malloc/free) is designed for general-purpose workloads, not deterministic latency. It may take a lock, search free lists, or request pages from the kernel via mmap. Even the fastest allocator introduces jitter. A real-world Rust matching engine benchmark showed that replacing Vec::new() with a reusable global buffer reduced latency spikes and produced more consistent sub-100μs performance . The fix is not a faster allocator—it is no allocator on the hot path.

The cache coherence problem. When two threads communicate through shared memory, the hardware must transfer cache lines between cores. On Intel processors, this transfer takes approximately 40–55 nanoseconds . A naive queue that touches two cache lines per message (one for the data, one for a shared control variable) doubles the floor. The solution is to co-locate the synchronization metadata with the payload and pad indices to 64-byte boundaries to eliminate false sharing .

The lock contention problem. A mutex or atomic compare-and-swap on a shared counter serializes access. In HFT, the producer and consumer run on different cores. The best design is single-producer single-consumer (SPSC): the producer owns the write index, the consumer owns the read index, and the only shared state is the data itself . The Rust type system enforces this: Producer and Consumer are distinct types, and only one of each can exist .

The arena failure problem. A bump allocator like bumpalo is often proposed as a solution for allocation latency. It allocates by incrementing a pointer within a pre-allocated chunk, which is extremely fast . But in one HFT benchmark, switching to bumpalo “didn’t help or regressed”—the default allocator was already optimized, and the arena added complexity without benefit . The lesson is that arena allocation is valuable when allocations happen in phases and can be deallocated in bulk. On a hot path that reuses a fixed set of objects, an object pool or pre-allocated buffer is simpler and faster.

The trade-off. Zero-allocation design requires careful pre-planning. The maximum number of orders in flight, the size of the market data queue, and the number of price levels must all be known before the trading session begins. This is not a limitation—it is a feature. Deterministic memory usage means deterministic latency.

a. Pre-allocation and object pooling

The hot path must never call the system allocator. Every object that will be used during trading is allocated at startup and reused. An object pool is the simplest pattern: a pre-allocated Vec of objects, with a free list tracking which are available.

struct OrderPool {
    orders: Vec<Order>,
    free: Vec<usize>,
}

impl OrderPool {
    fn new(capacity: usize) -> Self {
        let orders = (0..capacity).map(|_| Order::default()).collect();
        let free = (0..capacity).rev().collect();
        Self { orders, free }
    }

    fn acquire(&mut self) -> Option<&mut Order> {
        let idx = self.free.pop()?;
        Some(&mut self.orders[idx])
    }

    fn release(&mut self, idx: usize) {
        self.free.push(idx);
    }
}

This pattern eliminates allocation on the hot path. The pool is created once, and every acquire is a simple pop from the free list. The order is not deallocated when released; it is returned to the pool for reuse.

For variable-sized messages, a ring buffer of fixed-size slots is the standard approach. The market data handler in a low-latency Rust portfolio uses a UDP receive loop that decodes messages into a pre-allocated buffer, with zero allocations on the hot path .

b. Lock-free SPSC queues with cache-line alignment

The inter-thread communication channel is the most latency-sensitive structure in the system. A mutex is unacceptable. An MPSC channel with internal locking is unacceptable. The correct primitive is an SPSC ring buffer with cache-aligned indices.

#[repr(align(64))]
struct CachePadded<T>(T);

struct SpscQueue<T, const N: usize> {
    buffer: [UnsafeCell<MaybeUninit<T>>; N],
    head: CachePadded<AtomicUsize>,  // Producer writes
    tail: CachePadded<AtomicUsize>,  // Consumer writes
}

The head and tail indices are separated by at least one cache line to prevent false sharing. The producer updates head with Release ordering after writing data; the consumer loads head with Acquire before reading . This ensures the data is visible before the index update.

The qring crate implements exactly this design: lock-free, zero-allocation, cache-aligned, with compile-time capacity . The photon-ring crate goes further with a seqlock-stamped slot design that co-locates the sequence stamp with the payload, reducing the read path to a single cache-line fetch for payloads up to 56 bytes . Benchmarks show 48 ns p50 one-way latency on an Intel i7-10700KF .

c. Struct layout and cache locality

Even with pre-allocated memory, the layout of objects in memory determines how many cache misses occur. A linked list of orders scattered across the heap will produce a cache miss on every traversal. A contiguous array of orders, sorted by price, will have the next order in the same or adjacent cache line.

The matching engine benchmark that replaced a B-tree with a sorted Vec and binary search saw latency drop from ~50μs to ~20–30μs . The B-tree nodes were scattered; the Vec was contiguous. The improvement was cache locality, not algorithmic complexity.

For the limit order book, the price levels can be stored in a dense array indexed by price tick. If the tick size is 0.01 and the price range is 100.00 to 110.00, that is 1000 slots—small enough to fit in L1 or L2 cache. The order at each level is a linked list of indices into the pre-allocated order pool, not pointers.

struct PriceLevel {
    head: Option<usize>,  // Index into order pool
    tail: Option<usize>,
    volume: u64,
}

struct OrderBook {
    bids: Vec<PriceLevel>,  // Dense array, indexed by tick
    asks: Vec<PriceLevel>,
}

This layout is the difference between a cache miss on every match and a cache hit. In a low-latency portfolio, the limit order book engine achieves p99 add latency of ≈60 ns and p99 cancel latency of ≈80 ns with zero heap allocations on the hot path .

Complete Example Session

# ============================================
# PART 1: PRE-ALLOCATED ORDER POOL
# ============================================

struct Order {
    id: u64,
    price: u64,
    quantity: u64,
    next: Option<usize>,
}

struct OrderPool {
    orders: Vec<Order>,
    free: Vec<usize>,
}

impl OrderPool {
    fn with_capacity(cap: usize) -> Self {
        Self {
            orders: (0..cap).map(|_| Order::default()).collect(),
            free: (0..cap).rev().collect(),
        }
    }
}

# ============================================
# PART 2: CACHE-ALIGNED SPSC QUEUE
# ============================================

#[repr(align(64))]
struct CachePadded<T>(T);

struct Spsc<T, const N: usize> {
    slots: [UnsafeCell<MaybeUninit<T>>; N],
    head: CachePadded<AtomicUsize>,
    tail: CachePadded<AtomicUsize>,
}

// Producer:
unsafe fn push(&self, value: T) -> Result<(), T> {
    let head = self.head.0.load(Relaxed);
    let next = (head + 1) % N;
    if next == self.tail.0.load(Acquire) {
        return Err(value); // full
    }
    unsafe { (*self.slots[head].get()).write(value); }
    self.head.0.store(next, Release);
    Ok(())
}

# ============================================
# PART 3: DENSE PRICE LEVEL ARRAY
# ============================================

const MIN_PRICE: u64 = 10000; // 100.00
const MAX_PRICE: u64 = 11000; // 110.00

struct Book {
    bids: Vec<Level>,
    asks: Vec<Level>,
}

impl Book {
    fn new() -> Self {
        let len = (MAX_PRICE - MIN_PRICE) as usize;
        Self {
            bids: vec![Level::default(); len],
            asks: vec![Level::default(); len],
        }
    }
    
    fn bid_level(&mut self, price: u64) -> &mut Level {
        &mut self.bids[(price - MIN_PRICE) as usize]
    }
}

# ============================================
# PART 4: REUSABLE TRADE BUFFER
# ============================================

// Global buffer, cleared but not deallocated.
static mut TRADE_BUFFER: Vec<Trade> = Vec::new();

fn on_trade(trade: Trade) {
    unsafe { TRADE_BUFFER.push(trade); }
    // Periodic flush to strategy, then clear.
    // Capacity is never released.
}

# ============================================
# PART 5: AVOIDING SYSTEMTIME
# ============================================

// SystemTime::now() is a syscall (vDSO) with variable latency.
// Use a monotonic atomic counter for IDs.
static ORDER_ID: AtomicU64 = AtomicU64::new(1);

fn next_id() -> u64 {
    ORDER_ID.fetch_add(1, Relaxed)
}

# ============================================
# PART 6: SEQLOCK-STAMPED SLOT
# ============================================

#[repr(C, align(64))]
struct Slot<T: Copy> {
    stamp: AtomicU64,
    value: MaybeUninit<T>,
}

// Publisher:
fn publish<T: Copy>(slot: &Slot<T>, seq: u64, value: T) {
    slot.stamp.store(seq * 2 + 1, Relaxed); // odd = writing
    fence(Release);
    unsafe { (*slot.value.as_ptr()).write(value); }
    slot.stamp.store(seq * 2 + 2, Release); // even = done
}

# ============================================
# PART 7: PINNING TO CORE (WITH CAVEATS)
# ============================================

// Pinning can help, but benchmarks show it can regress.
// Test with your workload before committing.
// use core_affinity::set_for_current(core_id);

# ============================================
# PART 8: ZERO-ALLOCATION MARKET DATA HANDLER
# ============================================

// UDP socket receives into pre-allocated buffer.
let mut buf = [0u8; 1500];
loop {
    let n = socket.recv(&mut buf)?;
    let msg = decode(&buf[..n]); // Zero-copy decode
    queue.push(msg)?;
}

# ============================================
# PART 9: LATENCY MEASUREMENT
# ============================================

// Measure at p50, p90, p99, p99.9.
// A single average hides the tail that matters.
// Use rdtsc or clock_gettime with a pre-allocated histogram.

The nine parts trace the optimization landscape: pre-allocated object pools, cache-aligned SPSC queues, dense price-level arrays, reusable buffers, avoiding system calls, seqlock-stamped slots, the caveat about core pinning, zero-allocation market data handling, and percentile-based latency measurement.

Quick Reference

Zero-Allocation Patterns

Pattern	Use
Object pool	Fixed-size orders, reusable
SPSC ring buffer	Inter-thread communication
Dense array	Price levels, indexed by tick
Reusable buffer	Trade accumulation, logging

Cache Alignment

Structure	Alignment
Producer index	64-byte cache line
Consumer index	64-byte cache line (separate)
Slot payload	64-byte cache line (co-located with stamp)

Latency Targets (from benchmarks)

Operation	Target
Order add (p99)	≈60 ns
Order cancel (p99)	≈80 ns
SPSC one-way (p50)	48 ns

Best Practices

✅ Do This:

let pool = OrderPool::with_capacity(100_000);
// Pre-allocate everything before trading starts.

✅ Every object allocated at startup

#[repr(align(64))]
struct CachePadded<T>(T);
// Separate producer and consumer indices.

✅ Eliminate false sharing

static ORDER_ID: AtomicU64 = AtomicU64::new(1);
// Use atomic counter, not SystemTime::now().

✅ Avoid variable-latency syscalls

❌ Don’t Do This:

let order = Box::new(Order::new());
// Allocation on the hot path.

❌ Unpredictable latency

let lock = Mutex::new(queue);
// Lock contention on inter-thread channel.

❌ Serializes producer and consumer

// Assume bumpalo will fix allocation latency.

❌ Benchmark showed no benefit or regression

Common Pitfalls

Pitfall	Why It Happens	Fix
Latency spikes	Allocator called on hot path	Pre-allocate, use object pool
Inconsistent latency	`SystemTime::now()` in hot loop	Atomic counter for IDs
High inter-thread latency	False sharing on indices	Cache-line padding
Cache misses on match	Linked list of orders	Contiguous sorted Vec
Bumpalo regression	Arena overhead without benefit	Reusable buffer or pool

Real-World Examples

1. Pre-allocated Order Pool

let pool = OrderPool::with_capacity(100_000);

2. Cache-Aligned SPSC

#[repr(align(64))]
struct CachePadded<T>(T);

3. Dense Price Levels

let level = &mut book.bids[(price - MIN) as usize];

4. Reusable Trade Buffer

static mut TRADE_BUFFER: Vec<Trade> = Vec::new();

5. Atomic ID

static ORDER_ID: AtomicU64 = AtomicU64::new(1);

6. Seqlock Publish

slot.stamp.store(seq * 2 + 1, Relaxed);

7. Zero-Copy Decode

let msg = decode(&buf[..n]);

8. Percentile Measurement

// Measure p99, not average.

9. Sorted Vec for Book

// Replaced BTree with sorted Vec + binary search.

10. SPSC Queue

use qring::spsc::SpscQueue;

Visual

Hot Path Allocation Boundaries

┌──────────────────────────────────────────────┐
│  STARTUP (allocations allowed)               │
│                                              │
│  Order pool: 100,000 orders                  │
│  SPSC queue: 65,536 slots                    │
│  Price levels: 1,000 slots                   │
│  Trade buffer: 10,000 slots                  │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  HOT PATH (zero allocations)                 │
│                                              │
│  on_tick() {                                 │
│    pool.acquire() -> no alloc                │
│    queue.push() -> no alloc                  │
│    book.match() -> no alloc                  │
│  }                                           │
└──────────────────────────────────────────────┘

The boundary between startup and hot path is the most important design decision.

SPSC Cache Line Layout

┌──────────────────────────────────────────────┐
│  Cache line 1 (producer core)                │
│  head: AtomicUsize                           │
│  padding: [56 bytes]                         │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Cache line 2 (consumer core)                │
│  tail: AtomicUsize                           │
│  padding: [56 bytes]                         │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Cache lines 3+ (data)                       │
│  slots: [UnsafeCell<MaybeUninit<T>>; N]      │
└──────────────────────────────────────────────┘

The indices are on separate cache lines. The producer never invalidates the consumer’s cache line.

Summary

Item	Value
Hot path allocation	Zero
Inter-thread channel	Lock-free SPSC, cache-aligned
Order storage	Pre-allocated pool
Price levels	Dense array, indexed by tick
IDs	Atomic counter, not `SystemTime`
Cache line padding	64 bytes for shared indices
Benchmark target	p99 ≈60 ns for order operations

Key takeaways:

The hot path must be allocation-free. Every object is pre-allocated at startup. The system allocator is never called during trading .
SPSC queues eliminate lock contention. The producer and consumer own separate indices. Cache-line padding prevents false sharing .
Cache locality dominates algorithmic complexity. A sorted Vec outperformed a B-tree because the data was contiguous .
Arena allocators are not a universal fix. bumpalo did not improve latency in a matching engine benchmark; the default allocator was already optimized for the workload .
System calls introduce jitter. Replacing SystemTime::now() with an atomic counter reduced latency spikes .

Remember: HFT optimization is not about writing faster code. It is about removing non-deterministic operations. Allocation, locking, and system calls all introduce variance. The hot path is deterministic only when every operation is bounded and every memory access is predictable.

Was this helpful?

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!