Rust 78 🦀 Abstract Syntax Tree Manipulation (syn and quote crates) - KandZ

⏱️ Estimated read: 15 min | |

A procedural macro receives a flat stream of tokens. It cannot ask “is this a struct or an enum?” or “what are the field names?” at the token level. The syn crate parses that token stream into a typed syntax tree—DeriveInput, ItemFn, Expr, Type—so the macro can navigate the structure with Rust’s pattern matching and exhaustiveness checking. The quote crate does the inverse: it takes a template of Rust code with interpolated variables and produces a token stream. Together, they are the read and write API for procedural macros, and they are the foundation of serde, clap, tokio, and virtually every non-trivial derive macro in the ecosystem .

The division of labor is precise. syn answers “what is this code?” and quote answers “how do I write this code?” A macro that generates a getter for every field uses syn to iterate the fields and extract their names and types, then uses quote! to build the impl block with those names and types interpolated. The macro author never constructs tokens by hand; the parser produces the syntax tree, the template produces the output, and the runtime handles the conversion.

Key point: syn parses a TokenStream into a typed AST (DeriveInput, ItemFn, etc.); quote! generates a proc_macro2::TokenStream from a Rust-shaped template with #var interpolation and #(...)* repetition. The entry point is syn::parse_macro_input!, the output is converted with .into(), and errors are reported with syn::Error::new(span, message).to_compile_error().

Why these two crates exist

The token problem. A TokenStream is a flat sequence of TokenTree values—identifiers, punctuation, literals, and delimited groups. There is no structure beyond the delimiters. To know whether a sequence of tokens is a struct declaration or a function call, the macro must parse it. Writing that parser by hand is a full-time project. syn provides the parser and the syntax tree for every Rust construct, feature-gated so the macro only pays for what it uses .

The construction problem. Generating code by hand means constructing TokenStream values one token at a time: an Ident for the name, a Punct for the colon, a Group for the braces. This is verbose and error-prone. quote! lets the macro author write what the output should look like, with #variable interpolation for the parts that come from the parsed input. The result is code that reads like the code it generates .

The span problem. An error in a procedural macro is useless if it points at the macro invocation instead of the offending field. syn attaches a Span to every parsed token, and syn::Error::new(field.span(), "message") produces a compile error that underlines the exact field. quote! propagates spans through interpolation, so a type that came from the user’s code keeps its original location in the generated code .

The testability problem. The compiler’s proc_macro types cannot exist outside a procedural macro. They cannot be used in unit tests, in a main.rs, or in a build.rs. quote! produces proc_macro2::TokenStream, which can exist anywhere. This is why the ecosystem is built on proc_macro2: the macro logic can be unit-tested without invoking the compiler, and the conversion to proc_macro::TokenStream happens only at the boundary with .into() .

a. Parsing with `syn`: the `DeriveInput` and the `Parse` trait

The most common entry point is parse_macro_input!(input as DeriveInput). The DeriveInput type represents any of the three legal inputs to a derive macro: a struct, an enum, or a union. It has fields for the ident (the type name), the generics, the attrs, and the data (which distinguishes struct from enum from union) .

// Import the parser entry point and the DeriveInput type
use syn::{parse_macro_input, DeriveInput, Data, Fields};
// Import the token stream type from the compiler
use proc_macro::TokenStream;

// The derive macro entry point
#[proc_macro_derive(Getters)]
pub fn derive_getters(input: TokenStream) -> TokenStream {
    // Parse the input tokens into a DeriveInput syntax tree
    // If the input is not a valid item, this returns a compile error
    let ast = parse_macro_input!(input as DeriveInput);

    // Extract the type name from the parsed tree
    let name = &ast.ident;

    // Match on the data to handle only structs with named fields
    let fields = match &ast.data {
        // The Data::Struct variant contains the field definitions
        Data::Struct(data) => match &data.fields {
            // The Fields::Named variant contains the named fields
            Fields::Named(fields) => &fields.named,
            // Reject tuple structs with a spanned error
            _ => return syn::Error::new(ast.ident.span(), "expected named fields")
                .to_compile_error()
                .into(),
        },
        // Reject enums and unions
        _ => return syn::Error::new(ast.ident.span(), "expected a struct")
            .to_compile_error()
            .into(),
    };

    // The rest of the macro generates the getters from the fields
    // (shown in the next section)
    quote::quote! {}.into()
}

The parse_macro_input! macro is the standard parser entry point. It parses the token stream into the specified type and returns a compile_error! token stream if the parse fails, so the error points at the macro invocation rather than producing a panic . The ast.data field is an enum, and matching on it is how the macro distinguishes a struct from an enum. The Fields::Named variant is the one that has the ident for each field; tuple and unit structs do not.

For types that do not have a default parsing behavior, syn exposes the Parse trait and the Parser trait. The Parser trait provides parse, parse2, and parse_str methods, and it is implemented by the parser functions for types like Punctuated that need to know whether trailing punctuation is allowed . This is how a macro that parses a custom syntax—a comma-separated list of expressions, for example—defines its own grammar.

b. Generating with `quote!`: interpolation and repetition

The quote! macro produces a proc_macro2::TokenStream from a template. The template looks like Rust code, but the # prefix marks a variable to interpolate, and #(...)* marks a repetition.

// Import the quote macro for generating tokens
use quote::quote;

// Inside the derive macro, after extracting the fields
// Build a getter for each field by mapping over the field list
let getters = fields.iter().map(|field| {
    // The field name as an identifier
    let field_name = field.ident.as_ref().unwrap();
    // The field type, used as the return type of the getter
    let field_type = &field.ty;
    // Generate one method per field
    // #field_name and #field_type are interpolated into the template
    quote! {
        // The generated method returns a reference to the field
        pub fn #field_name(&self) -> &#field_type {
            // The field access uses the same identifier
            &self.#field_name
        }
    }
});

// Collect the getters and build the impl block
// #(#getters)* repeats the getter tokens for every field
let expanded = quote! {
    impl #name {
        // The repetition inserts each generated method
        #(#getters)*
    }
};

The #name interpolation inserts the parsed type name into the impl header. The #(#getters)* repetition iterates over the getters iterator and inserts each TokenStream it produces. The #field_name and #field_type interpolations inside the mapped closure insert the field’s name and type into the method signature and body .

The repetition syntax mirrors macro_rules!. #(#var)* inserts the elements with no separator. #(#var),* inserts them separated by commas, without a trailing comma. #(#var ,)* inserts them separated by commas with a trailing comma . The distinction matters when the generated code is a list of items where a trailing comma would be a syntax error.

The output of quote! is a proc_macro2::TokenStream, and the procedural macro must return a proc_macro::TokenStream. The conversion is .into() at the end of the function. If the macro has helper functions that build parts of the output, their quote! results are also proc_macro2::TokenStream and can be interpolated into later quote! calls because TokenStream implements ToTokens .

c. Attributes and nested meta parsing

A derive macro that reads helper attributes—like #[skip] or #[serde(rename = "x")]—must parse the attribute’s metadata. syn provides Attribute::parse_nested_meta for the structured attribute syntax and meta.path.is_ident("name") for checking the attribute’s path.

// Parse a field's attributes to check for #[skip]
// field.attrs is a Vec<Attribute>
for attr in &field.attrs {
    // Check whether the attribute is the one we registered
    if attr.path().is_ident("skip") {
        // The attribute is #[skip]; mark the field as skipped
        // (In a real macro, this would set a flag or filter the field)
        println!("field is skipped");
    }
}

// Parse a structured attribute like #[tea(kind = "EarlGrey", with(sugar, milk))]
// parse_nested_meta takes a closure that receives each nested meta item
if attr.path().is_ident("tea") {
    // The closure is called once per comma-separated item inside the parens
    attr.parse_nested_meta(|meta| {
        // Check for the "kind" key
        if meta.path.is_ident("kind") {
            // Parse the value after the equals sign as a string literal
            let value = meta.value()?;
            let s: syn::LitStr = value.parse()?;
            // Use the parsed string
            println!("kind = {}", s.value());
        // Check for the "with" list
        } else if meta.path.is_ident("with") {
            // Parse the nested list recursively
            meta.parse_nested_meta(|inner| {
                // Each inner item is an identifier like "sugar" or "milk"
                if inner.path.is_ident("sugar") {
                    println!("with sugar");
                }
                Ok(())
            })?;
        }
        Ok(())
    })?;
}

The parse_nested_meta method is the modern replacement for the old parse_meta and NestedMeta types, which were removed in syn 2.0 . The method takes a closure that receives a ParseNestedMeta, which has a path (the key) and an input stream (the rest of the item). The meta.value() method advances past the = and returns the stream for the value, which can be parsed as a LitStr, an Ident, or any other Parse type .

The helper attribute must be declared in the derive macro’s registration: #[proc_macro_derive(Describe, attributes(skip))]. Without that declaration, the compiler rejects the #[skip] attribute as unknown before the macro runs . The attributes(...) argument is the mechanism that allows the macro to define its own attribute namespace.

Complete Example Session

The following session builds a derive macro that generates getters for struct fields, skips fields marked with #[skip], and threads the type’s generics through the generated impl.

// ============================================
// PART 1: THE MACRO CRATE SETUP
// ============================================
// [lib]
// proc-macro = true
//
// [dependencies]
// syn = { version = "2", features = ["full"] }
// quote = "1"

// ============================================
// PART 2: THE IMPORTS
// ============================================
use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput, Data, Fields};

// ============================================
// PART 3: THE DERIVE MACRO
// ============================================
// Declare the derive macro and its helper attribute
#[proc_macro_derive(Getters, attributes(skip))]
pub fn derive_getters(input: TokenStream) -> TokenStream {
    // Parse the input tokens into a DeriveInput
    let ast = parse_macro_input!(input as DeriveInput);

    // Extract the type name
    let name = &ast.ident;

    // Split the generics into the three parts needed for the impl header
    // Without this, generic types would fail to compile
    let (impl_generics, ty_generics, where_clause) = ast.generics.split_for_impl();

    // Extract the named fields from the struct
    let fields = match &ast.data {
        Data::Struct(data) => match &data.fields {
            Fields::Named(fields) => &fields.named,
            _ => return syn::Error::new(ast.ident.span(), "expected named fields")
                .to_compile_error()
                .into(),
        },
        _ => return syn::Error::new(ast.ident.span(), "expected a struct")
            .to_compile_error()
            .into(),
    };

    // Filter out skipped fields and generate a getter for each remaining field
    let getters = fields
        .iter()
        .filter(|field| {
            // Keep the field only if it does not have #[skip]
            !field.attrs.iter().any(|attr| attr.path().is_ident("skip"))
        })
        .map(|field| {
            // The field name as an identifier
            let field_name = field.ident.as_ref().unwrap();
            // The field type as a type node
            let field_type = &field.ty;
            // Generate one getter method per field
            quote! {
                // The method returns a reference to the field
                pub fn #field_name(&self) -> &#field_type {
                    // Access the field by its identifier
                    &self.#field_name
                }
            }
        });

    // Build the impl block with the generics threaded through
    let expanded = quote! {
        // The impl header uses the split generics
        impl #impl_generics #name #ty_generics #where_clause {
            // The repetition inserts each generated getter
            #(#getters)*
        }
    };

    // Convert the proc_macro2 token stream to the compiler's token stream
    expanded.into()
}

// ============================================
// PART 4: THE USAGE
// ============================================
// Import the derive macro from its crate
use getters_derive::Getters;

// The derive generates getters for name and age
// The #[skip] attribute excludes password
#[derive(Getters)]
struct User {
    name: String,
    age: u32,
    #[skip]
    password: String,
}

fn main() {
    let user = User {
        name: "Alice".into(),
        age: 30,
        password: "secret".into(),
    };
    // The generated getters are callable
    println!("{} is {}", user.name(), user.age());
    // user.password() does not exist
}

The four parts cover the crate setup, the imports, the macro implementation with generics and attribute filtering, and the usage. What the session demonstrates is that syn parses the struct into a navigable tree, quote! generates the methods from the field list, and the helper attribute excludes the password field from the generated API.

Quick Reference

syn Types

Type	Purpose
`DeriveInput`	Parsed struct, enum, or union
`ItemFn`	Parsed function (attribute macros)
`Expr`	Parsed expression (function-like macros)
`Type`	Parsed type
`Ident`	Parsed identifier
`Fields`	Struct field container
`Attribute`	Parsed attribute

Parsing Functions

Function	Purpose
`parse_macro_input!`	Parse with automatic error
`syn::parse`	Parse a `proc_macro::TokenStream`
`syn::parse2`	Parse a `proc_macro2::TokenStream`
`syn::parse_str`	Parse a string
`Parser::parse`	Parse with a custom parser

quote! Syntax

Syntax	Meaning
`#var`	Interpolate a variable
`#(#var)*`	Repeat, no separator
`#(#var),*`	Repeat, comma-separated
`#(#var ,)*`	Repeat, comma with trailing

Error Handling

Function	Purpose
`syn::Error::new(span, msg)`	Create a spanned error
`.to_compile_error()`	Convert to token stream
`meta.error(msg)`	Error from nested meta

Best Practices

✅ Do This:

// Parse with parse_macro_input!
let ast = parse_macro_input!(input as DeriveInput);

// Thread generics through the impl
let (impl_g, ty_g, where_c) = ast.generics.split_for_impl();

// Use quote! for generation
quote! { impl #impl_g Trait for #name #ty_g #where_c { ... } }

// Report errors with spans
syn::Error::new(ast.ident.span(), "expected struct").to_compile_error()

❌ Don’t Do This:

// Ignore generics
impl #name { ... } // fails for generic types

// Panic on bad input
panic!("not a struct") // no span, useless error

// Build tokens by hand
// TokenStream::from_iter([...]) // verbose and error-prone

Common Pitfalls

Pitfall	Why It Happens	Fix
Generic impl fails	Generics not threaded	`split_for_impl()`
Parse error has no span	Used `panic!`	`syn::Error`
Attribute rejected	Helper not declared	`attributes(name)`
Repetition produces wrong separators	Used `#(#var)` instead of `#(#var),`	Match the separator to the output
Macro not testable	Used `proc_macro` types	Use `proc_macro2` internally

Real-World Examples

1. Parse a derive input

let ast = parse_macro_input!(input as DeriveInput);

2. Extract the type name

let name = &ast.ident;

3. Split generics

let (impl_g, ty_g, where_c) = ast.generics.split_for_impl();

4. Generate an impl

quote! { impl #impl_g Trait for #name #ty_g #where_c { ... } }

5. Repeat over fields

#(#getters)*

6. Comma-separated repetition

#(#fields),*

7. Parse a nested meta

attr.parse_nested_meta(|meta| { ... })

8. Check a path

meta.path.is_ident("skip")

9. Spanned error

syn::Error::new(field.span(), "error").to_compile_error()

10. Convert output

expanded.into()

Visual

The syn/quote Pipeline

┌──────────────────────────────────────────────┐
│  TokenStream (from compiler)                 │
│         │                                    │
│         ▼                                    │
│  syn::parse_macro_input!                     │
│         │                                    │
│         ▼                                    │
│  Typed AST (DeriveInput, ItemFn, Expr)       │
│         │                                    │
│         ▼                                    │
│  quote! template                             │
│  #var interpolation, #(...)* repetition      │
│         │                                    │
│         ▼                                    │
│  proc_macro2::TokenStream                    │
│         │                                    │
│         ▼                                    │
│  .into() → proc_macro::TokenStream           │
└──────────────────────────────────────────────┘

Interpolation

┌──────────────────────────────────────────────┐
│  quote! {                                    │
│    impl #name {                              │
│      pub fn #field_name(&self) -> &#ty {     │
│        &self.#field_name                     │
│      }                                       │
│    }                                         │
│  }                                           │
│         │                                    │
│         ▼                                    │
│  #name, #field_name, #ty replaced            │
│  with the parsed values                      │
└──────────────────────────────────────────────┘

Repetition

┌──────────────────────────────────────────────┐
│  #(#getters)*                                │
│         │                                    │
│         ▼                                    │
│  getters = [tokens1, tokens2, tokens3]       │
│         │                                    │
│         ▼                                    │
│  tokens1 tokens2 tokens3                     │
│  (no separator)                              │
│                                              │
│  #(#fields),*                                │
│  → fields[0], fields[1], fields[2]           │
└──────────────────────────────────────────────┘

Summary

Item	Value
Parser crate	`syn`
Generator crate	`quote`
Parser entry	`parse_macro_input!`
Common AST type	`DeriveInput`
Generator macro	`quote!`
Interpolation	`#var`
Repetition	`#(...)*`
Error type	`syn::Error`
Output type	`proc_macro2::TokenStream`
Conversion	`.into()`

Key takeaways:

syn parses, quote! generates. syn turns the token stream into a typed AST that the macro can navigate with pattern matching; quote! turns a Rust-shaped template into a token stream with interpolation and repetition .
DeriveInput is the derive macro’s entry point. It contains the type name, the generics, and the data that distinguishes struct from enum. Matching on ast.data is how the macro handles each case .
split_for_impl() threads the generics. A derive macro that ignores generics produces an impl that fails for generic types. The method returns the three pieces needed for the impl header .
quote! produces proc_macro2::TokenStream for testability. The compiler-specific types cannot exist outside a macro; proc_macro2 can, so the macro logic can be unit-tested .
Errors must carry spans. syn::Error::new(span, message).to_compile_error() points the compiler error at the offending code; a panic! produces an error with no useful location .
Helper attributes must be declared. The attributes(name) argument on proc_macro_derive registers the attribute so the compiler does not reject it before the macro runs .

Remember: syn and quote are the read and write halves of the procedural macro. syn takes the flat token stream and gives it structure—a DeriveInput with an ident, a generics, and a data that can be matched. quote! takes a template that looks like the code you want to generate and fills in the parsed pieces. The pipeline is always the same: parse with parse_macro_input!, navigate the AST, build with quote!, and convert with .into(). The generics must be threaded, the errors must be spanned, and the output must be testable. These two crates are not language features—they are ecosystem crates—but they are the foundation of every non-trivial macro in the Rust ecosystem.

Was this helpful?

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!