The Allocator

The idea in one minute#

When a value escapes, the runtime’s allocator finds it memory. It is built for one goal: make the common case — a small object — lock-free and a few nanoseconds.

It does that with three ideas. Sizes are rounded up to one of about 70 size classes, so a free slot of the right class fits exactly and fragmentation is bounded. Memory is carved into spans — runs of 8 KB pages — each dedicated to one size class. And every scheduler processor (P) has a private cache of spans, the mcache, so allocating usually touches no lock at all.

An analogy#

A hardware shop’s screws. Loose screws are not sold one size per customer request; they come in standard sizes, each in its own bin. Every cashier keeps a small tray of each size at their till and serves customers from it without leaving the counter. When a tray is empty the cashier swaps it at the stock room, and when the stock room is out, it orders a pallet from the warehouse.

A picture#

flowchart TB
  REQ["mallocgc(size, type)"] --> SZ{"size?"}
  SZ -->|"under 16 B, no pointers"| TINY["Tiny allocator<br/>pack several into one 16 B block"]
  SZ -->|"up to 32 KB"| CLS["Round up to a size class<br/>8, 16, 24, 32, 48, 64, 80 ... 32768"]
  SZ -->|"over 32 KB"| LARGE["Large: dedicated span<br/>straight from the heap"]
  CLS --> MC["mcache of the current P<br/>one span per class, NO LOCK"]
  MC -->|"span has a free slot"| OBJ["return the slot, zeroed"]
  MC -->|"span full"| CEN["mcentral for that class<br/>locked: swap for a span with free slots"]
  CEN -->|"none available"| HEAP["mheap<br/>locked: carve a span from free pages"]
  HEAP -->|"no free pages"| OS["OS: map more memory<br/>in 64 MB arenas"]
  LARGE --> HEAP
  class REQ neutral
  class SZ queue
  class TINY,CLS,MC,OBJ compute
  class CEN,HEAP,LARGE memory
  class OS io

How it really works#

Size classes#

Requests up to 32 KB are rounded up to a class: 8, 16, 24, 32, 48, 64, 80, 96, 112, 128, … 32768 bytes. Classes are close together at small sizes and spaced about 12.5% apart at larger ones, so the rounding waste is bounded — proportionally largest for small odd sizes (a 33-byte object occupies 48).

This is the origin of advice in II.04: trimming a struct from 33 bytes to 32 saves 16 bytes per object, and it is why append capacities look odd (848, 1280) — growth is rounded up to what the allocator will hand out anyway.

Each class comes in two variants: scan (contains pointers) and noscan (does not). The garbage collector never looks inside noscan spans.

Spans, pages and arenas#

• The heap is reserved from the OS in large arenas (64 MB on 64-bit Linux) and divided into 8 KB pages.
• A span is a run of pages serving one size class. A span for 32-byte objects is one page holding 256 slots; a bitmap records which are in use.
• Because every object in a span is the same size, a pointer anywhere inside an object can be mapped to the object’s start and size with arithmetic — which the GC relies on.

Three tiers#

The fast path — a free slot in the mcache’s span — is a bitmap lookup and a pointer bump. Only when a span is exhausted does a goroutine visit mcentral, and only when mcentral is empty does it reach mheap. Go 1.27 added size-specialized allocation routines that make the fast path for objects under 80 bytes up to 30% cheaper.

Tiny and large objects#

• Tiny allocator: pointer-free objects smaller than 16 bytes (small strings, single numbers) are packed several to a 16-byte block. Cheap, but the block is freed only when every tiny object in it is dead.
• Large objects (over 32 KB) bypass the caches and get their own span, rounded up to whole 8 KB pages (a 33,000-byte request occupies 40,960). They are more expensive to allocate and to zero — a 1 MB buffer allocated per request is a classic performance problem; reuse it (lesson 05).

Zeroing#

Go guarantees memory is zeroed. Fresh pages from the OS are already zero; reused memory is cleared at allocation. For large buffers, zeroing is a significant part of the cost — another reason to reuse them.

Returning memory to the OS#

Freed spans go back to the heap for reuse. A background scavenger returns pages that stay unused to the operating system (MADV_DONTNEED/MADV_FREE on Linux), gradually, and more eagerly when a memory limit is set. So after a spike, the process’s resident memory falls over seconds to minutes, not instantly. debug.FreeOSMemory() forces it.

Reading the numbers#

HeapInuse − HeapAlloc is fragmentation and slack. The process’s RSS as the OS reports it is roughly Sys − HeapReleased. runtime/metrics exposes the same data with stable names and is what exporters use (see Observability II.02).

Code#

// alloc.go — size classes, the tiny allocator, and what a large allocation costs.
package main

import (
	"fmt"
	"runtime"
	"testing"
	"unsafe"
)

var sink []byte

// actualSize finds how many bytes the allocator really hands out for a request, by allocating
// many objects and dividing the heap growth.
func actualSize(n int) float64 {
	const count = 4096
	keep := make([][]byte, count)
	var before, after runtime.MemStats
	runtime.GC()
	runtime.ReadMemStats(&before)
	for i := range keep {
		keep[i] = make([]byte, n)
	}
	runtime.ReadMemStats(&after)
	runtime.KeepAlive(keep)
	return float64(after.HeapAlloc-before.HeapAlloc) / count
}

type P1 struct{ a, b, c, d int64 } // 32 bytes
type P2 struct {
	a, b, c, d int64
	e          byte
} // 33 → padded to 40 → size class 48

func main() {
	fmt.Println("requested → actually allocated (bytes)")
	for _, n := range []int{1, 8, 9, 16, 17, 33, 49, 65, 100, 129, 500, 1025, 5000, 33000} {
		got := actualSize(n)
		fmt.Printf("  %6d → %8.0f   (%.0f%% over)\n", n, got, 100*(got-float64(n))/float64(n))
	}

	fmt.Printf("\nsizeof P1 = %d, sizeof P2 = %d: one extra byte costs a whole size class\n",
		unsafe.Sizeof(P1{}), unsafe.Sizeof(P2{}))

	// Cost by size: small is a few ns; large pays for zeroing and a trip to the heap lock.
	fmt.Println("\nallocation cost")
	for _, n := range []int{16, 256, 4096, 32 << 10, 64 << 10, 1 << 20} {
		r := testing.Benchmark(func(b *testing.B) {
			for i := 0; i < b.N; i++ {
				sink = make([]byte, n)
			}
		})
		fmt.Printf("  %8d bytes: %8.0f ns  (%.2f ns per kB)\n", n,
			float64(r.T.Nanoseconds())/float64(r.N), float64(r.T.Nanoseconds())/float64(r.N)/(float64(n)/1024))
	}

	var m runtime.MemStats
	runtime.ReadMemStats(&m)
	fmt.Printf("\nHeapAlloc %d kB  HeapInuse %d kB  HeapIdle %d kB  HeapReleased %d kB  Sys %d kB\n",
		m.HeapAlloc/1024, m.HeapInuse/1024, m.HeapIdle/1024, m.HeapReleased/1024, m.Sys/1024)
	fmt.Printf("objects allocated so far: %d, freed: %d\n", m.Mallocs, m.Frees)
}

Remember this#

• Small allocations are rounded up to a size class and served from a per-P cache with no lock.
• Memory is organized as arenas → 8 KB pages → spans of one size class each.
• Objects over 32 KB take a slower path and are expensive to zero. Reuse big buffers.
• HeapAlloc is your live heap; RSS is roughly Sys − HeapReleased, and falls slowly after a spike.

Try it#

1. Run alloc.go. Plot requested versus actual size. Where is the rounding waste largest?
2. Find a struct in your own code and check whether it sits just above a size-class boundary.
3. Allocate a 1 MB buffer per iteration in a loop, then reuse one buffer instead. Compare time and TotalAlloc.

Check yourself#

1. Why does the allocator round sizes up to classes?
2. Which tier is lock-free, and why can it be?
3. Why is a large allocation disproportionately expensive?