niche.rs about

nichy is a visualizer for Rust type memory layouts. It uses rustc's internal layout engine to show you exactly how your types are arranged in memory, including field ordering, padding bytes, alignment, and niche optimization.

Rust object layout

Every value in Rust occupies a fixed number of bytes determined by its type. The compiler decides the size (total bytes), alignment (what byte boundary the value must start on), and field order for each type.

Alignment and padding

CPUs access memory most efficiently when values sit at addresses that are multiples of their alignment. A u32 has alignment 4, so it should start at an address divisible by 4. A struct's alignment is the maximum alignment of its fields.

The compiler inserts padding bytes between fields (and after the last field) to satisfy alignment constraints. These bytes exist in memory but carry no data.

struct Padded {
    a: u8,   // 1 byte
    b: u64,  // 8 bytes, align 8
    c: u8,   // 1 byte
}

Layout representations

Enum layout

Enums need to store which variant is active (the discriminant) alongside the variant's data. The compiler uses two strategies:

Tagged explicit discriminant

In the straightforward approach, the compiler reserves extra bytes for a tag that records which variant is active. The tag size depends on how many variants the enum has: 1 byte covers up to 256 variants, 2 bytes cover more, and so on.

enum Shape {
    Circle(f64),       // radius
    Rect(f64, f64),    // width, height
    Point,
}

Niche optimized zero-cost discrimination

This is the optimization that gives nichy its name. Some types have invalid bit patterns, meaning values the type can never hold. These unused patterns are called niches, and the compiler can repurpose them to encode the discriminant, eliminating the extra tag entirely.

Niche optimization in detail

A niche is a range of bit patterns that a type guarantees it will never use as a valid value. The compiler tracks niches through the type system and exploits them when building enum layouts.

Where niches come from

Niches propagate upward through structs. If a struct contains a bool, the struct itself exposes those 254 niches to any enum wrapping it.

The classic example: Option<&T>

type Ref = Option<&u64>;

Multiple niche values

When a type has more niches than an enum needs, the extras are remaining niches that propagate further outward. For example, bool has 254 niches. Option<bool> uses one (storing None as the byte value 2), leaving 253. So Option<Option<bool>> consumes one more and is still just 1 byte. An enum with up to 254 data-less variants wrapping a bool can fit in a single byte the same way.

This stacking works as long as niches remain. Types with only a single niche, like &T, support one layer of Option at no cost, but Option<Option<&T>> exhausts the supply and requires a tag.

When niche optimization cannot apply

• No niches available. Types like u32 or f64 use all their bit patterns, so wrapping them in Option requires a tag byte.
• Not enough niches. One variant is stored directly; every additional variant needs a niche. If there aren't enough, the enum falls back to tagged layout.
• #[repr(C)] or #[repr(u8)] enums. Explicit representation attributes force a predictable tagged layout.

Why this matters

Niche optimization is one of Rust's most impactful zero-cost abstractions. It means Option<Box<T>>, Option<&T>, and Option<NonZeroU32> are all the same size as the inner type. This removes the performance penalty for using Option instead of nullable pointers, making safe APIs free at the machine level.

The optimization is applied transitively. A Vec<Option<&T>> stores its elements at 8 bytes each (on 64-bit), the same density as a Vec<*const T> with a manual null convention.

Common misunderstandings

Niche optimization is well-known among Rust programmers, but its limitations are often misunderstood. Here are cases where it might seem like niche optimization should help, but doesn't.

Niches can't encode variant data

When the compiler uses a niche value to represent a variant, it uses that specific bit pattern as the discriminant. This only works for variants that carry no data of their own, or whose data fits elsewhere in the enum's payload (#46213). In the common case, such as Option::None, the niche-encoded variant has no fields at all, which is why the optimization works so cleanly.

enum Either {
    Left(NonNull<u8>),
    Right(NonNull<u8>),
}

Padding bytes are not niches

Struct padding exists for alignment but is not tracked as niche space. Even when a struct has trailing padding that could physically hold a discriminant, the compiler doesn't repurpose it (#70230, #109555). Niches come from type-level value constraints (like "references are never null") rather than from incidental unused bytes in the memory layout.

type A = Option<WithNiche>;
type B = Option<WithoutNiche>;

struct WithNiche {
    ptr: NonNull<u64>,
    flag: u8,
    // 7 bytes of trailing padding
}

struct WithoutNiche {
    ptr: *mut u64,
    flag: u8,
    // 7 bytes of trailing padding
}

Niche optimization only applies when it reduces size

Even when an enum's variants contain types with usable niches, the compiler only uses niche encoding when it makes the overall layout smaller. If the discriminant already fits in alignment padding, niche encoding produces the same size and the compiler picks a tagged layout instead. Whether niche optimization applies can therefore depend on the target triple.

enum Expr {
    Literal(f64),
    BinOp { op: char, lhs: Box<f64>, rhs: Box<f64> },
    Neg(Box<f64>),
}

What the niche system actually tracks

The compiler tracks niches as value ranges that a type guarantees it will never use. This is a type-level property rather than a layout-level one:

• NonNull<T> → value 0 is invalid (1 niche), regardless of T's alignment
• bool → values 2..=255 are invalid (254 niches)
• &T → value 0 is invalid (1 niche)
• Alignment-derived bit constraints → not tracked
• Padding bytes → not tracked

If you want to verify whether niche optimization applies to your types, paste them into the visualizer. A niche optimized badge means the compiler found usable niches; a tagged badge means it fell back to an explicit discriminant.

Related articles

• Type Layout — The Rust Reference

  The authoritative specification for how Rust lays out types in memory, covering repr(Rust), repr(C), primitive representations, and alignment rules.

• Niche Optimizations in Rust — 0xAtticus

  Deep dive into how the compiler identifies niches and encodes enum discriminants, with examples traced through LLVM IR and rustc source.

• Data Representation in Rust — The Rustonomicon

  Practical guide to repr(Rust), repr(C), and other representations, oriented toward unsafe code and FFI.

• Option Representation — std docs

  Documents the guaranteed null-pointer optimization for Option<&T>, Option<Box<T>>, and similar types.

• Visualizing Memory Layout of Rust's Data Types — YouTube

  Visual walkthrough of how Rust's common types (structs, enums, references, trait objects) are laid out in stack and heap memory.

• Type Sizes — The Rust Performance Book

  Actionable advice on shrinking types to improve cache performance, including #[repr(u8)] enums, Box-ing large variants, and field reordering.

• A Surprising Enum Size Optimization — James Fennell

  How the Rust compiler reuses tag bits from nested enums to reduce memory overhead, going beyond the well-known niche optimization.

^* The surrogate range 0xD800..=0xDFFF is also invalid for char, but rustc currently does not use those values as niches.