Rust 107 🦀 Compiler Internals: Abstract Syntax Tree (AST) to High-Level Intermediate Representation (HIR)
The Rust compiler does not type-check the syntax tree you write. Between the parser and the type checker sits a lowering step that converts the Abstract Syntax Tree (AST) into the High-Level Intermediate Representation (HIR). This step is not a refactoring for cleanliness. It is where syntactic sugar is dissolved, where for loops become loop plus match, where impl Trait becomes a generic parameter, and where the compiler decides which identifiers are definitions and which are references. The HIR is the first representation that is “amenable to type checking” , and everything the borrow checker later does begins with the structure this step produces.
This chapter covers the AST-to-HIR lowering pipeline as it exists in the rustc_ast_lowering crate. It begins with the desugarings that the step performs—the constructs that disappear from the tree—and the invariants that the lowering code must uphold to avoid tripping the compiler’s own sanity checks. It then addresses the HirId and the owner model that make incremental compilation and query-based analysis possible, and the distinction between the AST’s NodeId and the HIR’s HirId. The final sections cover the entry point, the LoweringContext, and the reason this step exists as a separate crate rather than as a pass in the parser.
Key point: The HIR is the AST after the syntax has been reduced to semantics. Parentheses are gone, for loops are gone, and impl Trait is gone. What remains is the smallest tree that type checking and borrow checking need.
Why lowering exists
The syntax-semantics gap. The AST preserves everything the user wrote, including constructs that are meaningful to the parser but irrelevant to the type checker. Parentheses are a parsing artifact; the tree structure already encodes precedence. A for loop is a convenience for iterating an iterator; the type checker only needs the Iterator::next calls and the loop structure, not the syntax. Lowering removes these constructs so that later passes do not have to handle them .
The desugaring responsibility. Some constructs are defined in terms of other constructs. A for loop is defined as a match on the result of IntoIterator::into_iter, inside a loop. An if let is a match with a single arm and a fallback. An async block is a generator, and an impl Trait in argument position is a generic parameter. The lowering step performs these definitions, so that the type checker sees a smaller language than the surface syntax .
The identity model. The AST uses NodeId to identify syntax nodes, but NodeId is a parser artifact. The HIR introduces HirId, which is composed of an OwnerId (the enclosing item, trait item, or impl item) and an ItemLocalId (unique within that owner) . This two-level identity is what makes the query system and incremental compilation practical: a change to one function body does not invalidate the HIR of every other function, because each owner is lowered independently.
The boundary for macros. Macro expansion happens before lowering. The AST that lowering receives is the post-expansion AST, with the macro invocations already replaced by their output . This means that the HIR does not represent macro calls as syntax; it represents the expanded code. The NodeId that lowering maps to a HirId is the post-expansion NodeId, not the pre-expansion one.
a. The desugarings: what disappears
The lowering step removes or rewrites a specific list of constructs. The rustc-dev-guide enumerates them, and the list is the most concrete description of what the HIR actually contains .
Parentheses are removed without replacement. The tree structure already makes the evaluation order explicit, so the parentheses that the user wrote to disambiguate precedence are redundant in the HIR.
for loops are converted to loop plus match. The iterator is obtained with IntoIterator::into_iter, and each iteration calls next. A None result breaks the loop. The for syntax does not exist in the HIR.
while let loops are converted to loop plus match, in the same way. The pattern match on the let expression determines whether the loop continues.
if let is converted to match. The single arm handles the pattern, and the fallback handles the non-matching case.
Universal impl Trait (in argument position) is converted to a generic argument. The HIR records a flag indicating that the user did not write the generic parameter explicitly, so that error messages and diagnostics can distinguish the two forms .
Existential impl Trait (in return position) is converted to a virtual existential type declaration. The type is represented as an opaque type with a hidden underlying type, and the declaration is synthesized during lowering.
The pattern is consistent: the HIR is smaller than the AST, and the constructs that remain are the ones that type checking and borrow checking need.
b. The invariants: HirId, owners, and the sanity checks
Lowering is not a free-form transformation. The compiler enforces a set of invariants through hir_id_validator.rs, and violating them produces a panic during compilation. The invariants are documented in the lowering guide, and they are the practical rules for anyone modifying the lowering code .
A HirId must be used if it is created. If lowering calls lower_node_id, the resulting NodeId or HirId must be placed into the HIR. Creating an ID and discarding it is a bug, because the validator checks that every NodeId in the HIR has a corresponding HirId.
Lowering a HirId must be done in the scope of the owning item. Each item, trait item, and impl item is lowered in its own with_hir_id_owner scope. If lowering creates parts of an item other than the one currently being lowered, it must enter the correct owner’s scope first. Existential impl Trait is the example: the synthesized type declaration belongs to the item that contains the impl Trait, not to the function being lowered.
A NodeId placed into the HIR must be lowered, even if its HirId is unused. The call let _ = self.lower_node_id(node_id); is the idiom for this case. The validator does not care whether the HirId is used; it cares whether it exists.
New nodes must get new IDs. If lowering creates a node that did not exist in the AST—a synthesized match arm, a desugared loop—it must call next_id, which produces both a new NodeId and a new HirId. Reusing an AST NodeId for a synthesized node would confuse the validator and the incremental compilation system.
The owner model is the reason for the third and fourth invariants. The HIR is organized into owners, and each owner’s body is a separate unit for incremental compilation and query caching. An ID that belongs to one owner cannot be used in another owner’s body without breaking the model.
c. The entry point, the context, and the crate
The lowering step is implemented in rustc_ast_lowering. The entry point is lower_to_hir, which retrieves the post-expansion AST and the resolver data from TyCtxt and produces the hir::Crate for the whole crate .
The lower_to_hir function first indexes the crate, then calls ItemLowerer::lower_node for each crate-level item, associated item, and foreign item. The actual lowering logic lives on LoweringContext, a struct that carries the state for the current owner and the ID-lowering machinery. The implementation is split across files—item.rs, expr.rs, pat.rs, path.rs—but they all share the same context .
The LoweringContext holds the mapping from AST NodeId to HIR HirId, the current owner, and the stack of scopes. The lower_node_id method maps an AST ID to a HIR ID in the current owner. The next_id method creates a fresh ID for a synthesized node. The with_hir_id_owner method enters a new owner’s scope, which is required when lowering a nested item or a synthesized declaration .
The output is the hir::Crate, the top-level structure that stores the entire contents of the crate. The HIR is arena-allocated, and the crate is the root of the arena. The HirId values that lowering produces are the keys into the arena, and the owner model determines which arena the values live in.
Complete Example Session
// ============================================
// PART 1: THE INPUT AST
// ============================================
// The AST for a for loop contains the iteration
// syntax. The HIR does not.
// AST (conceptual):
// ExprKind::ForLoop(pat, iter, body, label)
// ============================================
// PART 2: THE DESUGARED HIR
// ============================================
// The for loop is converted to loop + match.
// HIR (conceptual):
// loop {
// match IntoIterator::into_iter(iter) {
// Some(pat) => { body }
// None => break,
// }
// }
// ============================================
// PART 3: THE HIR OWNER
// ============================================
// Each item, trait item, and impl item is an owner.
// The owner's body is lowered independently.
// The HirId is composed of:
// OwnerId: the enclosing item's LocalDefId
// ItemLocalId: unique within the owner
// ============================================
// PART 4: THE LOWERING CONTEXT
// ============================================
// LoweringContext carries the state for the current owner.
// struct LoweringContext<'a, 'hir> {
// tcx: TyCtxt<'hir>,
// resolver: &'a Resolver<'a>,
// current_hir_id_owner: OwnerId,
// // ... ID mapping, scopes, etc.
// }
// ============================================
// PART 5: THE ENTRY POINT
// ============================================
// lower_to_hir indexes the crate and lowers each owner.
// fn lower_to_hir(tcx: TyCtxt<'_>) -> &hir::Crate<'_> {
// // Index the crate.
// // For each owner: lower_node(owner)
// // Build the hir::Crate.
// }
// ============================================
// PART 6: THE ID MAPPING
// ============================================
// lower_node_id maps AST NodeId to HIR HirId.
// next_id creates a fresh ID for synthesized nodes.
// let hir_id = self.lower_node_id(ast_node_id);
// let new_hir_id = self.next_id();
// ============================================
// PART 7: THE OWNER SCOPE
// ============================================
// with_hir_id_owner enters a new owner's scope.
// Required when lowering nested items.
// self.with_hir_id_owner(owner_id, |lctx| {
// // Lower the nested item.
// });
// ============================================
// PART 8: THE INVARIANTS
// ============================================
// The validator checks that every NodeId in the HIR
// has a corresponding HirId, and that IDs are used
// in the correct owner scope.
// Violations produce a panic during compilation.
// ============================================
// PART 9: THE OUTPUT
// ============================================
// The hir::Crate is the arena-allocated HIR.
// The type checker and borrow checker consume it.
// ============================================
// PART 10: THE FULL PICTURE
// ============================================
// The AST is syntax. The HIR is semantics.
// Lowering is the transformation between them.
// The step is a separate crate because it is
// a distinct responsibility with its own invariants.The ten parts build a conceptual view of the lowering step: the input AST, the desugared HIR, the owner model, the context, and the invariants. The pattern is consistent: the AST carries syntax, the HIR carries semantics, and lowering is the boundary where the syntax is reduced.
Quick Reference
Desugarings
| AST Construct | HIR Representation |
|---|---|
| Parentheses | Removed |
for loop | loop + match |
while let | loop + match |
if let | match |
Universal impl Trait | Generic argument (flagged) |
Existential impl Trait | Virtual existential type |
Identity Types
| ID | Scope | Created By |
|---|---|---|
NodeId | Crate-wide | Parser |
HirId | Owner-local | Lowering |
OwnerId | Crate-wide | Lowering |
ItemLocalId | Owner-local | Lowering |
Lowering API
| Function | Purpose |
|---|---|
lower_to_hir | Entry point, produces hir::Crate |
ItemLowerer::lower_node | Lower a single owner |
LoweringContext::lower_node_id | Map AST NodeId to HIR HirId |
LoweringContext::next_id | Create fresh ID for synthesized node |
LoweringContext::with_hir_id_owner | Enter an owner’s scope |
Invariants
| Invariant | Consequence of Violation |
|---|---|
HirId created must be used | Validator panic |
| Lowering in correct owner scope | Validator panic |
| Synthesized nodes get new IDs | Validator panic |
NodeId in HIR must be lowered | Validator panic |
Best Practices
✅ Do This:
// Use lower_node_id for AST nodes.
let hir_id = self.lower_node_id(node_id); // ✅// Use next_id for synthesized nodes.
let new_id = self.next_id(); // ✅// Enter the owner scope for nested items.
self.with_hir_id_owner(owner, |lctx| { ... }); // ✅// Lower every NodeId placed into the HIR.
let _ = self.lower_node_id(node_id); // ✅❌ Don’t Do This:
// Don't discard a created HirId.
self.lower_node_id(node_id); // unused // ❌// Don't reuse an AST NodeId for a synthesized node.
// Use next_id() instead. // ❌// Don't lower a nested item without entering its owner scope.
// The validator will panic. // ❌Common Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
Validator panic: unused HirId | ID created but not placed in HIR | Use the returned HirId |
| Validator panic: wrong owner | Nested item lowered in parent scope | with_hir_id_owner |
Validator panic: missing HirId | NodeId in HIR not lowered | let _ = lower_node_id(...) |
Duplicate DefId | Reusing NodeId for synthesized node | next_id() |
Real-World Examples
1. Desugar for
// loop { match iter.next() { Some(p) => body, None => break } }2. Desugar if let
// match expr { pat => body, _ => else_body }3. Desugar Universal impl Trait
// fn foo(x: impl Trait) → fn foo<T: Trait>(x: T)4. Owner Scope
self.with_hir_id_owner(owner_id, |lctx| { ... });5. ID Mapping
let hir_id = self.lower_node_id(node_id);6. Fresh ID
let new_id = self.next_id();7. Entry Point
lower_to_hir(tcx) -> &hir::Crate8. HIR Crate
// The arena-allocated HIR for the whole crate.9. Validator
// compiler/rustc_passes/src/hir_id_validator.rs10. Lowering Crate
// rustc_ast_loweringVisual
The Pipeline
┌─────────────────────────────────────────────────────────┐
│ SOURCE │
│ │ │
│ ▼ parser │
│ AST (syntax preserved) │
│ │ │
│ ▼ macro expansion │
│ POST-EXPANSION AST │
│ │ │
│ ▼ rustc_ast_lowering │
│ HIR (semantics only) │
│ │ │
│ ▼ type checking │
│ TYPED HIR / THIR │
│ │ │
│ ▼ MIR construction │
│ MIR (control-flow graph) │
└─────────────────────────────────────────────────────────┘The Owner Model
┌─────────────────────────────────────────────────────────┐
│ CRATE │
│ ├── Item A (owner) │
│ │ └── HirId(owner=A, local=0..n) │
│ ├── Item B (owner) │
│ │ ├── HirId(owner=B, local=0..m) │
│ │ └── nested item → owner=C │
│ └── Item C (owner) │
│ │
│ Each owner is lowered independently. │
│ The HirId is scoped to its owner. │
└─────────────────────────────────────────────────────────┘The Desugaring
┌─────────────────────────────────────────────────────────┐
│ AST │
│ for x in iter { body } │
│ │ │
│ ▼ lower │
│ HIR │
│ loop { │
│ match IntoIterator::into_iter(iter) { │
│ Some(x) => body, │
│ None => break, │
│ } │
│ } │
│ │
│ The for syntax is gone. │
│ The loop and match are what the type checker sees. │
└─────────────────────────────────────────────────────────┘The ID Boundary
┌─────────────────────────────────────────────────────────┐
│ AST NodeId HIR HirId │
│ (parser) (lowering) │
│ │
│ Crate-wide unique Owner + local │
│ Syntax node identity Semantic identity │
│ │
│ lower_node_id maps between them. │
│ next_id creates fresh HIR IDs. │
└─────────────────────────────────────────────────────────┘Summary
| Item | Value |
|---|---|
| Crate | rustc_ast_lowering |
| Entry point | lower_to_hir |
| Output | hir::Crate |
| Context | LoweringContext |
| ID mapping | lower_node_id |
| Fresh ID | next_id |
| Owner scope | with_hir_id_owner |
| Validator | hir_id_validator.rs |
| Key desugarings | for, if let, impl Trait, parentheses |
Key takeaways:
- The HIR is the AST after syntax is reduced to semantics. Parentheses,
forloops, andimpl Traitdisappear or are rewritten. What remains is the tree that type checking and borrow checking need . - Lowering is organized around owners. Each item, trait item, and impl item is lowered independently, and the
HirIdis scoped to its owner. This is what makes incremental compilation and query caching practical . - The invariants are enforced by a validator. An unused
HirId, a wrong owner scope, or a missingHirIdproduces a panic. The rules exist to keep the owner model consistent. lower_node_idmaps AST IDs to HIR IDs.next_idcreates fresh IDs for synthesized nodes. The distinction matters because reusing an AST ID for a synthesized node breaks the model.- The step is a separate crate because it is a separate responsibility. The parser produces syntax, the lowering produces semantics, and the type checker consumes the result. Mixing them would make both harder to maintain.
Remember: The AST is what you wrote. The HIR is what the compiler understands. The lowering step is the transformation between them: it removes the syntax that has no semantic content, rewrites the constructs that are defined in terms of other constructs, and assigns the identities that the rest of the compiler uses. The owner model and the HirId are the structural decisions that make the compiler’s incremental and query-based architecture possible, and the validator is the enforcement mechanism that keeps those decisions consistent.
Stop using slow, ad-bloated tool sites! 🤮
🔎 Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)
⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free
🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!