Rust 106 🦀 Performance Profiling, Benchmarking (criterion), and SIMD Optimizations
Performance work in Rust follows a discipline that is easy to state and hard to practice: measure first, optimize the bottleneck, measure again. The tools that make this possible are the profilers that show where time is actually spent, the benchmarking frameworks that turn a measurement into a repeatable experiment, and the SIMD APIs that let the compiler process multiple values per instruction when the algorithm allows it. The order matters. A SIMD optimization applied to a function that is not the bottleneck is wasted effort, and a benchmark that the compiler constant-folds is worse than no benchmark at all.
This chapter covers the workflow from profiling to benchmarking to vectorization. It begins with the profilers that answer “where is the time going?”—flamegraphs for CPU hot paths and DHAT for allocation pressure—then explains the Criterion benchmarking framework and the black_box discipline that prevents the optimizer from eliminating the code under test. The final sections cover the three SIMD strategies: relying on LLVM’s auto-vectorization, using architecture-specific intrinsics with runtime detection, and the portable SIMD API that remains nightly-only.
Key point: A benchmark that the compiler optimizes away measures nothing. black_box is not optional. It is the difference between a measurement and a fiction.
Why performance work needs a workflow
The intuition problem. Programmers are bad at guessing where time goes. The code that looks expensive—a clone, a bounds check, a virtual dispatch—is often compiled away or hidden behind a memory access that dominates the profile. The Rust Performance Book states the principle directly: the biggest performance improvements often come from changes to algorithms or data structures, not low-level optimizations . Profiling is how you find out which is which.
The release-build problem. A debug build is not the program you ship. It has no optimizations, no inlining, and full debug assertions. Profiling a debug build shows the cost of the debug build, not the cost of the release build. Every profiling and benchmarking step must use --release, and the release profile should include debug symbols so the profiler can attribute work to source lines .
The measurement stability problem. A single run of a function is not a measurement. The CPU frequency scales, the cache state varies, and the operating system schedules other work. Criterion addresses this by running the benchmark many times, computing confidence intervals, and detecting outliers . The output is a statistical estimate, not a single number, and the estimate is what makes a before/after comparison meaningful.
a. Profiling: flamegraphs and allocation pressure
cargo-flamegraph is the simplest entry point. It wraps perf on Linux (and DTrace on macOS), samples the call stack at a fixed frequency, and produces an SVG where the width of each box is the proportion of samples that included that function . The installation is cargo install flamegraph, and the invocation is cargo flamegraph --bin myapp or cargo flamegraph --bench bench_name -- --bench . The resulting flamegraph.svg opens in a browser.
# Profile a binary. The --release flag is implied by cargo-flamegraph.
cargo flamegraph --bin myapp
# Profile a benchmark.
cargo flamegraph --bench my_benchmark -- --bench
# Increase sampling frequency for more detail (higher overhead).
cargo flamegraph --freq 999 --bin myappThe flamegraph’s y-axis is the call stack depth, and the x-axis is the sample space, not time. A wide box means the function consumed a large share of the samples, either because it is expensive per call or because it is called frequently . The common findings are malloc/free in the hot path (allocation pressure), memcpy (unnecessary copying), and small functions that are not being inlined .
perf on Linux provides more detail than the flamegraph wrapper. perf record --call-graph dwarf -F 99 target/release/app records with a 99 Hz sampling frequency and DWARF-based stack unwinding, which is more accurate than frame-pointer unwinding for optimized code. perf report --stdio produces a text report, and piping perf script through inferno-collapse-perf and inferno-flamegraph produces the same SVG . The perf_event_paranoid sysctl controls whether unprivileged users can profile; setting it to -1 permits it, at a security cost .
Allocation profiling answers a different question. When the flamegraph shows time in alloc:: or __rust_alloc, the question is not “which function allocates” but “why is allocation happening so often.” DHAT (dhat crate) is the Rust-native option. It requires a global allocator and a profiler guard, both behind a feature flag so the instrumentation does not affect release builds.
// Behind a feature flag so release builds are unaffected.
#[cfg(feature = "dhat-heap")]
#[global_allocator]
static ALLOC: dhat::Alloc = dhat::Alloc;
fn main() {
#[cfg(feature = "dhat-heap")]
let _profiler = dhat::Profiler::new_heap();
// ... application code
}The output is a dhat-heap.json file that the DHAT viewer reads, showing allocation counts, sizes, and lifetimes by call stack . A hot path that allocates many short-lived objects of the same size is a candidate for pre-allocation, a buffer pool, or a different data structure that avoids the allocation entirely.
b. Benchmarking with Criterion and black_box
Criterion is the stable-Rust replacement for the unstable #[bench] attribute. It runs each benchmark many times, computes the mean and confidence interval, detects outliers, and compares against a saved baseline to flag regressions . The setup requires a [[bench]] entry with harness = false and the criterion dev-dependency.
# Cargo.toml
[dev-dependencies]
criterion = { version = "0.5", features = ["html_reports"] }
[[bench]]
name = "my_benchmark"
harness = falseThe benchmark file lives in benches/. The criterion_group! and criterion_main! macros generate the entry point, and the Criterion argument provides the bench_function method.
use criterion::{criterion_group, criterion_main, Criterion};
use std::hint::black_box;
fn bench_fibonacci(c: &mut Criterion) {
c.bench_function("fib 20", |b| {
// black_box on the input prevents constant folding.
// black_box on the output prevents dead-code elimination.
b.iter(|| black_box(fibonacci(black_box(20))));
});
}
criterion_group!(benches, bench_fibonacci);
criterion_main!(benches);The black_box function is the critical detail. Without it, the compiler may evaluate fibonacci(20) at compile time and replace the benchmark loop with a constant. The benchmark then measures the cost of a constant, which is zero, and the report is meaningless . black_box tells the compiler that the value is opaque: it cannot be constant-folded, and its uses cannot be eliminated. The Rust standard library provides it as std::hint::black_box since Rust 1.66.
For input-size comparisons, bench_with_input and BenchmarkId group the results under a single name with different parameters. For function comparisons, multiple bench_function calls in the same group show the relative performance in the HTML report. The baseline commands—cargo bench -- --save-baseline v1 and cargo bench -- --baseline v1—persist a run and compare a later run against it .
use criterion::{BenchmarkId, Criterion};
fn bench_sort(c: &mut Criterion) {
let mut group = c.benchmark_group("sort");
for size in [100, 1_000, 10_000, 100_000] {
group.bench_with_input(
BenchmarkId::from_parameter(size),
&size,
|b, &size| {
b.iter(|| {
let mut v: Vec<i32> = (0..size).rev().collect();
v.sort();
black_box(v)
});
},
);
}
group.finish();
}c. SIMD: auto-vectorization, intrinsics, and portable SIMD
SIMD processes multiple values per instruction. A 256-bit AVX2 register holds eight i32 values, and a single vpaddd adds all eight pairs simultaneously. The Rust Performance Book and the language blog both note that the compiler’s auto-vectorizer can find these patterns, but it is easily confused and often fails on “obvious” vector tasks . The three strategies address this in increasing order of control and complexity.
Auto-vectorization (stable, no code changes). LLVM vectorizes loops that fit its pattern matchers: simple element-wise operations, no early exits, no aliasing that it cannot prove safe. The sum and add_arrays functions in the portable SIMD guide are the canonical examples . Writing the loop with iterators rather than indexing, and using chunks_exact for aligned access, improves the odds. The limitation is that the developer has no guarantee: the vectorizer may succeed on one compiler version and fail on the next, or succeed on x86_64 and fail on aarch64.
// May auto-vectorize.
fn add_arrays(a: &[f32], b: &[f32], out: &mut [f32]) {
for ((x, y), o) in a.iter().zip(b).zip(out.iter_mut()) {
*o = x + y;
}
}Architecture-specific intrinsics (stable, unsafe). The std::arch module exposes the platform’s SIMD instructions directly. The code is unsafe, the function names are platform-specific, and the implementation must be gated by target_feature. Runtime detection with is_x86_feature_detected!("avx2") allows a single binary to use the fastest available instruction set on the current CPU and fall back to scalar code otherwise .
#[cfg(target_arch = "x86_64")]
#[target_feature(enable = "avx2")]
unsafe fn add_avx2(a: &[f32], b: &[f32], out: &mut [f32]) {
use std::arch::x86_64::*;
// Load 8 floats, add, store.
// The unsafe block is required because the caller must
// guarantee the CPU supports AVX2 (checked by the dispatcher).
for i in (0..a.len()).step_by(8) {
let va = _mm256_loadu_ps(a.as_ptr().add(i));
let vb = _mm256_loadu_ps(b.as_ptr().add(i));
_mm256_storeu_ps(out.as_mut_ptr().add(i), _mm256_add_ps(va, vb));
}
}
fn add_dispatch(a: &[f32], b: &[f32], out: &mut [f32]) {
#[cfg(target_arch = "x86_64")]
if is_x86_feature_detected!("avx2") {
return unsafe { add_avx2(a, b, out) };
}
// Scalar fallback for other CPUs and architectures.
for ((x, y), o) in a.iter().zip(b).zip(out.iter_mut()) {
*o = x + y;
}
}Portable SIMD (nightly). std::simd provides types like f32x8 that map to the widest available SIMD register on the target, with a scalar fallback when no SIMD is available . The API is high-level, #![feature(portable_simd)] is required, and the code is not stable. The as_simd method splits a slice into a prefix, a middle that the SIMD loop processes, and a suffix, which is the correct way to handle unaligned edges . For stable Rust, the wide crate provides a similar abstraction over the platform’s SIMD without the nightly requirement .
#![feature(portable_simd)]
use std::simd::*;
fn sum_simd(data: &[f32]) -> f32 {
let (prefix, middle, suffix) = data.as_simd::<8>();
let mut sum = prefix.iter().sum::<f32>();
let mut simd_sum = f32x8::splat(0.0);
for chunk in middle {
simd_sum += *chunk;
}
sum + simd_sum.reduce_sum() + suffix.iter().sum::<f32>()
}Complete Example Session
// ============================================
// PART 1: THE BASELINE BENCHMARK
// ============================================
// Criterion setup with black_box on input and output.
use criterion::{criterion_group, criterion_main, Criterion};
use std::hint::black_box;
fn bench_baseline(c: &mut Criterion) {
c.bench_function("baseline", |b| {
b.iter(|| black_box(compute(black_box(100))));
});
}
// ============================================
// PART 2: THE INPUT-SIZE BENCHMARK
// ============================================
// Compare across sizes with BenchmarkId.
use criterion::BenchmarkId;
fn bench_sizes(c: &mut Criterion) {
let mut group = c.benchmark_group("process");
for size in [100, 1_000, 10_000] {
group.bench_with_input(
BenchmarkId::from_parameter(size),
&size,
|b, &size| {
b.iter(|| black_box(process(black_box(size))));
},
);
}
group.finish();
}
// ============================================
// PART 3: THE FUNCTION COMPARISON
// ============================================
// Two implementations under one group.
fn bench_compare(c: &mut Criterion) {
let mut group = c.benchmark_group("impl");
let input = black_box(vec![1u64; 10_000]);
group.bench_function("scalar", |b| {
b.iter(|| black_box(sum_scalar(&input)));
});
group.bench_function("simd", |b| {
b.iter(|| black_box(sum_simd(&input)));
});
group.finish();
}
// ============================================
// PART 4: THE FLAMEGRAPH
// ============================================
// cargo flamegraph --bench my_benchmark -- --bench
// Produces flamegraph.svg. Wide boxes are hot.
// ============================================
// PART 5: THE ALLOCATION PROFILE
// ============================================
// DHAT behind a feature flag.
#[cfg(feature = "dhat-heap")]
#[global_allocator]
static ALLOC: dhat::Alloc = dhat::Alloc;
// ============================================
// PART 6: THE SCALAR FALLBACK
// ============================================
// The reference implementation.
fn sum_scalar(data: &[u64]) -> u64 {
data.iter().sum()
}
// ============================================
// PART 7: THE AUTO-VECTORIZED LOOP
// ============================================
// Iterator-based, no early exit.
fn sum_auto(data: &[u64]) -> u64 {
data.iter().copied().fold(0, |a, b| a + b)
}
// ============================================
// PART 8: THE INTRINSIC VERSION
// ============================================
// AVX2 with runtime detection.
#[cfg(target_arch = "x86_64")]
#[target_feature(enable = "avx2")]
unsafe fn sum_avx2(data: &[u64]) -> u64 {
use std::arch::x86_64::*;
let mut acc = _mm256_setzero_si256();
let mut i = 0;
while i + 4 <= data.len() {
let v = _mm256_loadu_si256(data.as_ptr().add(i) as *const __m256i);
acc = _mm256_add_epi64(acc, v);
i += 4;
}
let mut lanes = [0u64; 4];
_mm256_storeu_si256(lanes.as_mut_ptr() as *mut __m256i, acc);
lanes.iter().sum::<u64>() + data[i..].iter().sum::<u64>()
}
// ============================================
// PART 9: THE DISPATCHER
// ============================================
// Choose the fastest available implementation.
fn sum_dispatch(data: &[u64]) -> u64 {
#[cfg(target_arch = "x86_64")]
if is_x86_feature_detected!("avx2") {
return unsafe { sum_avx2(data) };
}
sum_auto(data)
}
// ============================================
// PART 10: THE FULL PICTURE
// ============================================
// Profile first. Benchmark with black_box.
// Vectorize only the hot path.
criterion_group!(benches, bench_baseline, bench_sizes, bench_compare);
criterion_main!(benches);The ten parts build from the Criterion setup to the AVX2 dispatcher. The pattern is consistent: the benchmark uses black_box to prevent the compiler from eliminating the code, the flamegraph identifies the hot path, and the SIMD version is applied only where the profile justifies it.
Quick Reference
Profiling Tools
| Tool | Command | Answers |
|---|---|---|
cargo-flamegraph | cargo flamegraph --bin app | Where is CPU time spent? |
perf record | perf record --call-graph dwarf -F 99 ./app | Detailed stack attribution |
| DHAT | dhat crate + feature flag | Where is allocation pressure? |
perf report | perf report --stdio | Text hot-function list |
Criterion Setup
| Item | Value |
|---|---|
| Dependency | criterion = { version = "0.5", features = ["html_reports"] } |
| Harness | harness = false in [[bench]] |
| Entry | criterion_group! + criterion_main! |
| Prevent folding | black_box(input) and black_box(output) |
| Save baseline | cargo bench -- --save-baseline v1 |
| Compare | cargo bench -- --baseline v1 |
SIMD Strategies
| Strategy | Stability | Control | Portability |
|---|---|---|---|
| Auto-vectorization | Stable | None | LLVM-dependent |
std::arch intrinsics | Stable | Full | Per-architecture |
std::simd (portable) | Nightly | High | Cross-architecture |
wide crate | Stable | High | Cross-architecture |
Auto-Vectorization Rules
| Rule | Reason |
|---|---|
| Use iterators, not indexing | Bounds checks block vectorization |
| No early exits | Vectorizer requires uniform control flow |
Use chunks_exact | Aligned access, no remainder handling |
| Avoid aliasing | Vectorizer cannot prove independence |
Best Practices
✅ Do This:
// Use black_box on input and output.
b.iter(|| black_box(fibonacci(black_box(20)))); // ✅# Profile release builds with debug symbols.
[profile.release]
debug = true // ✅// Use runtime detection for intrinsics.
if is_x86_feature_detected!("avx2") { unsafe { avx2_impl() } } // ✅// Handle the SIMD prefix and suffix.
let (prefix, middle, suffix) = data.as_simd::<8>(); // ✅# Save a baseline before optimizing.
cargo bench -- --save-baseline before // ✅❌ Don’t Do This:
// Don't benchmark without black_box.
b.iter(|| fibonacci(20)); // constant-folded to zero // ❌# Don't profile a debug build.
cargo flamegraph # without --release // ❌// Don't apply SIMD without profiling.
sum_simd(&data); // if sum is 1% of runtime, no gain // ❌// Don't use intrinsics without a fallback.
#[target_feature(enable = "avx2")] // crashes on non-AVX2 CPUs // ❌// Don't assume auto-vectorization happened.
// Check the flamegraph or the assembly. // ❌Common Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
| Benchmark measures zero | Compiler constant-folds the result | Use black_box on input and output |
| Flamegraph has no symbols | Release build without debug info | Set debug = true in [profile.release] |
| SIMD crashes on older CPU | Intrinsic called without detection | Gate with is_x86_feature_detected! |
| SIMD slower than scalar | Setup cost exceeds loop cost | Profile first; SIMD is for hot loops |
| Allocation still hot | DHAT not run | Add the dhat feature flag and profile |
| Regression undetected | No saved baseline | --save-baseline before the change |
Real-World Examples
1. Flamegraph a Binary
cargo flamegraph --bin myapp2. Flamegraph a Benchmark
cargo flamegraph --bench my_bench -- --bench3. Criterion Benchmark
c.bench_function("name", |b| b.iter(|| black_box(f(black_box(x)))));4. Input Sizes
group.bench_with_input(BenchmarkId::from_parameter(size), &size, |b, &size| { ... });5. Save Baseline
cargo bench -- --save-baseline v16. Compare Baseline
cargo bench -- --baseline v17. AVX2 Detection
if is_x86_feature_detected!("avx2") { unsafe { f_avx2() } }8. Portable SIMD
#![feature(portable_simd)]
let (pre, mid, suf) = data.as_simd::<8>();9. DHAT Global Allocator
#[global_allocator]
static ALLOC: dhat::Alloc = dhat::Alloc;10. Release Debug Symbols
[profile.release]
debug = trueVisual
The Performance Workflow
┌─────────────────────────────────────────────────────────┐
│ 1. Write correct code │
│ │ │
│ ▼ │
│ 2. Profile (cargo flamegraph, perf) │
│ → identify the hot path │
│ │ │
│ ▼ │
│ 3. Benchmark the hot path (Criterion + black_box) │
│ → establish a baseline │
│ │ │
│ ▼ │
│ 4. Optimize ONE thing │
│ → algorithm, allocation, SIMD │
│ │ │
│ ▼ │
│ 5. Re-run the benchmark │
│ → compare against baseline │
│ │ │
│ ▼ │
│ 6. Keep or revert │
└─────────────────────────────────────────────────────────┘Reading a Flamegraph
┌─────────────────────────────────────────────────────────┐
│ Width = proportion of samples (time) │
│ Height = call stack depth │
│ │
│ ┌───────────────────────────────────────────────────┐ │
│ │ main │ │
│ │ ┌─────────────────────────────────────────────┐ │ │
│ │ │ process │ │ │
│ │ │ ┌──────────────┐ ┌────────────────────┐ │ │ │
│ │ │ │ clone │ │ expensive_compute │ │ │ │
│ │ │ │ (narrow) │ │ (WIDE = hot) │ │ │ │
│ │ │ └──────────────┘ └────────────────────┘ │ │ │
│ │ └─────────────────────────────────────────────┘ │ │
│ └───────────────────────────────────────────────────┘ │
│ │
│ Optimize the wide box, not the narrow one. │
└─────────────────────────────────────────────────────────┘black_box
┌─────────────────────────────────────────────────────────┐
│ WITHOUT black_box │
│ │
│ b.iter(|| fibonacci(20)) │
│ Compiler: "fibonacci(20) is a constant. I know it." │
│ → benchmark measures a constant load (0 ns) │
│ │
│ ───────────────────────────────────────────────────── │
│ │
│ WITH black_box │
│ │
│ b.iter(|| black_box(fibonacci(black_box(20)))) │
│ Compiler: "20 is opaque. The result is opaque." │
│ → benchmark measures the actual computation │
└─────────────────────────────────────────────────────────┘SIMD Dispatch
┌─────────────────────────────────────────────────────────┐
│ fn sum_dispatch(data: &[u64]) -> u64 │
│ │ │
│ ▼ is_x86_feature_detected!("avx2") │
│ ┌────┴────┐ │
│ ▼ ▼ │
│ YES NO │
│ │ │ │
│ ▼ ▼ │
│ sum_avx2 sum_auto │
│ (unsafe, (LLVM may │
│ 256-bit) vectorize) │
│ │
│ One binary, two code paths, runtime decision. │
└─────────────────────────────────────────────────────────┘Summary
| Item | Value |
|---|---|
| CPU profiler | cargo-flamegraph, perf record |
| Allocation profiler | dhat crate, DHAT viewer |
| Benchmarking | Criterion, harness = false |
| Prevent optimization | black_box(input) and black_box(output) |
| Baseline comparison | --save-baseline, --baseline |
| Auto-vectorization | LLVM, iterator-based loops |
| Intrinsics | std::arch, #[target_feature], is_x86_feature_detected! |
| Portable SIMD | std::simd (nightly), wide crate (stable) |
| Release debug info | debug = true in [profile.release] |
Key takeaways:
- Profile before optimizing. The bottleneck is rarely where intuition places it. The flamegraph shows the wide boxes, and the wide boxes are the only ones worth optimizing .
- Benchmark with
black_box. Without it, the compiler constant-folds the computation and the benchmark measures nothing. The input and the output must both be opaque to the optimizer . - Profile release builds with debug symbols. A debug build is not the program you ship, and a release build without symbols produces a flamegraph with no function names. Set
debug = truein[profile.release]. - Auto-vectorization is the first SIMD strategy. Iterator-based loops without early exits give LLVM the best chance. If the compiler succeeds, the code is portable and safe. If it fails, the flamegraph shows the scalar loop .
- Intrinsics require runtime detection and a fallback. The
is_x86_feature_detected!macro gates the AVX2 path, and the scalar path handles CPUs without it. Without the dispatcher, the binary crashes on older hardware . - Portable SIMD is nightly-only. The
std::simdAPI provides cross-architecture vectorization, but the#![feature(portable_simd)]attribute prevents stable compilation. Thewidecrate is the stable alternative . - Allocation pressure is a separate finding. When the flamegraph shows
mallocin the hot path, DHAT answers the question of which call site allocates and how long the allocations live. The fix is a different data structure or a pre-allocated buffer, not a faster allocator .
Remember: Performance work is an experiment, not an opinion. The flamegraph is the observation, the benchmark is the measurement, and the optimization is the hypothesis. black_box keeps the measurement honest. Runtime detection keeps the SIMD path safe. The workflow—profile, benchmark, optimize one thing, measure again—is the only part that does not change when the tools do.
Stop using slow, ad-bloated tool sites! 🤮
🔎 Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)
⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free
🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!