Generics and Iterators — Data in Memory

The idea in one minute#

Generics let a function or type take a type parameter: func Max[T cmp.Ordered](a, b T) T works for every ordered type, checked at compile time, with no any and no type assertions. A constraint — an interface — says what the type must support.

Iterators (Go 1.23) let for ... range loop over a function: any data structure can offer a sequence without first building a slice.

Use generics for containers and algorithms that are identical for every element type. Do not use them where an ordinary interface describes behaviour — that is still what interfaces are for.

An analogy#

A cookie cutter and a stamp. A generic function is a recipe written once with a blank for “the dough”; the kitchen makes a version for each dough you actually use. An iterator is a vending chute: it hands you one item at a time when you ask, instead of tipping the whole stock onto the counter first.

A picture#

flowchart TB
  SRC["func Sum[T Number](xs []T) T"] --> COMP["Compiler"]
  COMP --> S1["one copy per 'shape':<br/>all pointer types share one,<br/>int64 gets its own, float32 its own"]
  S1 --> DICT["plus a hidden dictionary argument<br/>describing the exact type"]
  COMP --> CHK["constraint checked at compile time:<br/>Sum[string] does not compile"]
  subgraph IT["Iterator: for v := range seq"]
    direction LR
    LOOP["loop body<br/>becomes a function 'yield'"] <-->|"seq calls yield(v) per element;<br/>yield returns false on break"| SEQ["seq func(yield func(V) bool)"]
  end
  class SRC neutral
  class COMP,CHK compute
  class S1,DICT memory
  class LOOP,SEQ queue

How it really works#

Type parameters and constraints#

type Number interface {
    ~int | ~int32 | ~int64 | ~float32 | ~float64      // a union of types; ~ includes named types built on them
}

func Sum[T Number](xs []T) T {
    var total T
    for _, x := range xs {
        total += x
    }
    return total
}

Sum([]float32{1, 2, 3})     // T inferred as float32

Constraint	Allows
`any`	Every type; you can only assign, pass and compare-to-nothing
`comparable`	Types usable with `==` and as map keys
`cmp.Ordered`	Types supporting `<`: integers, floats, strings
A union: `~int \| ~float64`	Operators those types share
An interface with methods	Types having those methods

Generic types#

type Stack[T any] struct{ items []T }

func (s *Stack[T]) Push(v T) { s.items = append(s.items, v) }
func (s *Stack[T]) Pop() (T, bool) {
    var zero T
    if len(s.items) == 0 {
        return zero, false
    }
    v := s.items[len(s.items)-1]
    s.items = s.items[:len(s.items)-1]
    return v, true
}

Until Go 1.27 a method could not introduce its own type parameters, so Map from Stack[T] to Stack[U] had to be a top-level function. Go 1.27 added generic methods:

func (s *Stack[T]) MapTo[U any](f func(T) U) *Stack[U] { /* ... */ }   // Go 1.27+

Interface methods still cannot declare type parameters.

How they are compiled, and what they cost#

Go does not generate a fully separate copy for every type argument, nor does it box everything. It groups type arguments by GC shape — essentially their memory layout:

Every distinct non-pointer shape (int64, float32, a particular struct) gets its own compiled copy: as fast as hand-written code.
All pointer types share one copy, which receives a hidden dictionary describing the actual type. Calling a method on a type parameter inside that copy goes through the dictionary — an indirect call that cannot be inlined, sometimes slower than an interface call.

So: generics over numeric and value types are free. Generics that call methods on a type parameter instantiated with pointer types may not be. Measure in a hot loop (V.01).

The standard generic packages#

Package	Has
`slices`	`Sort`, `SortFunc`, `BinarySearch`, `Contains`, `Index`, `Clone`, `Compact`, `Reverse`, `Max`, `Insert`, `Delete`
`maps`	`Keys`, `Values`, `Clone`, `Copy`, `DeleteFunc`
`cmp`	`Compare`, `Ordered`, `Or`
`sync` / `sync/atomic`	`atomic.Pointer[T]`, `sync.OnceValue`
`iter`	`Seq[V]`, `Seq2[K, V]`, `Pull`

slices.Sort is generic and noticeably faster than the old sort.Slice, which went through reflect and an interface.

Iterators: range over functions#

// iter.Seq[V] is:  func(yield func(V) bool)
func Tokens(text string) iter.Seq[string] {
    return func(yield func(string) bool) {
        for _, w := range strings.Fields(text) {
            if !yield(w) {      // false means the loop did `break` or returned
                return
            }
        }
    }
}

for tok := range Tokens("a b c") { /* ... */ }      // Go 1.23+

The compiler turns the loop body into the yield function. Simple iterators are inlined into the loop and cost nothing over a hand-written one. They compose — slices.Collect(maps.Keys(m)), slices.Sorted(seq), slices.Chunk(s, n) — and iter.Pull converts one into explicit next()/stop() calls when you must consume two sequences in step.

Iterators fit AI code well: a stream of tokens from a model, batches from a dataset, or search results are all “produce values until the consumer stops” (VI.06).

When to use what#

Situation	Reach for
A container or algorithm identical for all element types	Generics
Numeric code over `float32` and `float64`	Generics with a union constraint
Different types with different behaviour behind one API	An interface
A value that may be one of a few known types	A type switch
Producing a sequence lazily	An iterator

Code#

// generics.go — one generic function, three shapes, and what each costs.
package main

import (
	"cmp"
	"fmt"
	"slices"
	"testing"
)

type Number interface {
	~int | ~int64 | ~float32 | ~float64
}

func Sum[T Number](xs []T) T {
	var total T
	for _, x := range xs {
		total += x
	}
	return total
}

func SumFloat32(xs []float32) float32 { // the hand-written version, for comparison
	var total float32
	for _, x := range xs {
		total += x
	}
	return total
}

func SumAny(xs []any) float64 { // the pre-generics way: boxing and type switches
	total := 0.0
	for _, x := range xs {
		switch v := x.(type) {
		case float32:
			total += float64(v)
		case float64:
			total += v
		}
	}
	return total
}

func ArgMax[T cmp.Ordered](xs []T) int {
	best := 0
	for i, x := range xs {
		if x > xs[best] {
			best = i
		}
	}
	return best
}

// Stack is a generic type.
type Stack[T any] struct{ items []T }

func (s *Stack[T]) Push(v T) { s.items = append(s.items, v) }
func (s *Stack[T]) Pop() (v T, ok bool) {
	if len(s.items) == 0 {
		return v, false
	}
	v = s.items[len(s.items)-1]
	s.items = s.items[:len(s.items)-1]
	return v, true
}

// Map cannot be a method before Go 1.27, so it is a function.
func Map[T, U any](xs []T, f func(T) U) []U {
	out := make([]U, len(xs))
	for i, x := range xs {
		out[i] = f(x)
	}
	return out
}

func main() {
	logits := []float32{0.1, 2.5, -1.0, 2.4}
	fmt.Println("Sum:", Sum(logits), " ArgMax:", ArgMax(logits), " ArgMax of strings:", ArgMax([]string{"b", "z", "a"}))

	var st Stack[string]
	st.Push("prefill")
	st.Push("decode")
	top, _ := st.Pop()
	fmt.Println("popped:", top)
	fmt.Println("Map:", Map(logits, func(x float32) string { return fmt.Sprintf("%.1f", x) }))

	sorted := slices.Clone(logits)
	slices.Sort(sorted)
	fmt.Println("slices.Sort:", sorted)

	// Cost: generic vs hand-written vs []any.
	const n = 4096
	f := make([]float32, n)
	a := make([]any, n)
	for i := range f {
		f[i] = float32(i)
		a[i] = f[i]
	}
	var s32 float32
	var s64 float64
	bench := func(name string, fn func()) {
		r := testing.Benchmark(func(b *testing.B) {
			for i := 0; i < b.N; i++ {
				fn()
			}
		})
		fmt.Printf("  %-22s %6d ns per %d elements\n", name, r.NsPerOp(), n)
	}
	fmt.Println()
	bench("Sum[float32] generic", func() { s32 += Sum(f) })
	bench("SumFloat32 by hand", func() { s32 += SumFloat32(f) })
	bench("SumAny over []any", func() { s64 += SumAny(a) })
	_, _ = s32, s64
}

Iterators need Go 1.23, so they appear here as a snippet rather than in the runnable program:

// Batches yields consecutive chunks of at most n items without copying them.
func Batches[T any](xs []T, n int) iter.Seq[[]T] {
    return func(yield func([]T) bool) {
        for len(xs) > 0 {
            k := min(n, len(xs))
            if !yield(xs[:k:k]) {
                return
            }
            xs = xs[k:]
        }
    }
}

for batch := range Batches(prompts, 8) { run(batch) }

Remember this#

Type parameters with constraints give compile-time-checked reuse; no boxing for value types.
One compiled copy per memory shape; all pointer types share a copy and use a dictionary.
Generics for containers and numeric algorithms; interfaces for behaviour.
Iterators are functions you can range over: lazy, composable, and stoppable.

Try it#

Run generics.go. How does the generic Sum compare with the hand-written one? With []any?
Write TopK[T cmp.Ordered](xs []T, k int) []int returning the indexes of the k largest values. You will use it for sampling in module VI.
With Go 1.23 or newer, implement Batches and a Window[T any](xs []T, size int) iter.Seq[[]T] that yields overlapping windows. Break out of the loop early and confirm the iterator stops.

Check yourself#

What is a constraint, and what does ~ mean in one?
When can a generic function be slower than expected?
What does yield returning false mean inside an iterator?