Goroutines and the Scheduler

The idea in one minute#

A goroutine is a function running independently, with its own small stack, managed by the Go runtime rather than the operating system. Starting one costs a few kilobytes and well under a microsecond, so programs routinely run hundreds of thousands.

The runtime’s scheduler multiplexes them onto a small number of OS threads using three things: G (a goroutine), M (a machine: an OS thread), and P (a processor: a permission to run Go code, of which there are GOMAXPROCS). An M must hold a P to run a G. Each P has its own queue of runnable goroutines, and an idle P steals work from the others.

An analogy#

A restaurant kitchen. Orders (G) are slips of paper. Cooks (M) do the work. There is a fixed number of stoves (P): a cook can only cook at a stove. Each stove has its own rail of pending orders. A cook whose rail is empty takes half the slips from a busier stove. When a cook has to step out to wait for a delivery (a blocking system call), they leave the stove, and another cook takes it over so the stove never sits idle.

A picture#

flowchart TB
  subgraph P0["P0"]
    RQ0["local run queue<br/>up to 256 G"]
  end
  subgraph P1["P1"]
    RQ1["local run queue<br/>(empty)"]
  end
  GRQ["global run queue"]
  NP["netpoller<br/>G waiting on sockets, timers"]
  M0["M0: OS thread<br/>running G7"] --- P0
  M1["M1: OS thread<br/>looking for work"] --- P1
  M2["M2: blocked in a system call<br/>has handed its P away"]
  RQ1 -.->|"steal half"| RQ0
  M1 -.->|"else check"| GRQ
  M1 -.->|"else check"| NP
  SYS["sysmon: background thread<br/>preempts long-running G, retakes P from blocked M"]
  SYS -.-> M0
  SYS -.-> M2
  class RQ0,RQ1,GRQ queue
  class M0,M1 compute
  class M2 warn
  class NP io
  class SYS neutral

How it really works#

Goroutine versus thread#

go handle(conn)                  // start one
go func() { /* ... */ }()        // or an anonymous function

main returning ends the program without waiting for other goroutines. A goroutine has no identity you can use, no return value and no way to be killed from outside: it must be told to stop (lesson 03) and it must finish (lesson 06).

G, M, P#

• G — the goroutine: its stack, its saved registers when not running, its state.
• M — an OS thread. The runtime creates them as needed (default cap: 10,000).
• P — a scheduling context. There are exactly GOMAXPROCS of them, by default the number of CPUs available. Each P owns a local run queue (256 slots), an allocator cache (the mcache from III.03) and other per-CPU state.

GOMAXPROCS bounds how many goroutines execute Go code simultaneously. It does not bound goroutines, or threads (threads blocked in system calls do not hold a P).

Since Go 1.25, on Linux the default respects the container’s CPU limit (cgroup quota) and is updated if the limit changes. Before that, a container limited to 2 CPUs on a 64-core host ran 64 Ps, was throttled by the kernel, and showed latency spikes; the fix was to set GOMAXPROCS by hand or with a library.

The scheduling loop#

An M with a P looks for a goroutine to run, in roughly this order:

1. The P’s local run queue (plus one “run next” slot for the goroutine most recently made runnable — which is why a goroutine that wakes another usually sees it run immediately).
2. Every 61st time, the global run queue first, for fairness.
3. The global run queue.
4. The netpoller: goroutines whose network I/O became ready.
5. Work stealing: pick another P at random and take half of its local queue.

If there is nothing anywhere, the M parks. When a goroutine is made runnable and there is an idle P, an M is woken (or created) to pick it up.

When a goroutine stops running#

Consequences worth knowing#

• Goroutines are cheap, not free. A million parked goroutines is gigabytes of stack. More to the point, unbounded goroutine creation is unbounded concurrency: 100,000 simultaneous requests to a database is a different problem than scheduling (lesson 06).
• CPU-bound work does not speed up beyond GOMAXPROCS. For compute — a matrix multiply, a batch of embeddings — use about one goroutine per P, not one per item.
• File I/O and cgo block threads, unlike network I/O. Thousands of goroutines each in a cgo call or a slow disk read means thousands of OS threads.
• Order is not guaranteed. Any interleaving the memory model allows can happen.
• runtime.LockOSThread() pins a goroutine to its thread, for libraries that need thread-local state (some GPU and UI libraries).

Observing the scheduler#

GODEBUG=schedtrace=1000 ./app      # one line per second: runnable goroutines per P, idle Ps, threads
go tool trace trace.out            # per-goroutine timeline: running, runnable, blocked (V.01)

runtime.NumGoroutine() and the /sched/latencies:seconds metric in runtime/metrics (how long goroutines wait to run) are the two numbers to export.

Code#

// sched.go — goroutine cost, parallel speedup limited by GOMAXPROCS, and preemption.
package main

import (
	"fmt"
	"runtime"
	"sync"
	"sync/atomic"
	"time"
)

func burn(n int) uint64 { // CPU-bound work with no function calls and no allocation
	var x uint64 = 1
	for i := 0; i < n; i++ {
		x = x*6364136223846793005 + 1442695040888963407
	}
	return x
}

func parallel(workers, tasks, size int) time.Duration {
	start := time.Now()
	var wg sync.WaitGroup
	ch := make(chan int)
	for w := 0; w < workers; w++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for range ch {
				burn(size)
			}
		}()
	}
	for t := 0; t < tasks; t++ {
		ch <- t
	}
	close(ch)
	wg.Wait()
	return time.Since(start)
}

func main() {
	cpus := runtime.NumCPU()
	fmt.Println("CPUs:", cpus, " GOMAXPROCS:", runtime.GOMAXPROCS(0))

	// 1. Creating and finishing 100,000 goroutines.
	start := time.Now()
	var wg sync.WaitGroup
	for i := 0; i < 100000; i++ {
		wg.Add(1)
		go func() { wg.Done() }()
	}
	wg.Wait()
	fmt.Printf("100,000 goroutines created and finished: %.0f ns each\n",
		float64(time.Since(start).Nanoseconds())/100000)

	// 2. CPU-bound work scales with workers only up to GOMAXPROCS.
	const tasks, size = 64, 3_000_000
	base := parallel(1, tasks, size)
	fmt.Printf("\nworkers  time      speedup\n%7d  %7.0f ms   1.0x\n", 1, float64(base.Milliseconds()))
	for _, w := range []int{2, cpus, cpus * 8} {
		d := parallel(w, tasks, size)
		fmt.Printf("%7d  %7.0f ms  %4.1fx\n", w, float64(d.Milliseconds()), float64(base)/float64(d))
	}

	// 3. Limit to one P: goroutines still interleave, because the runtime preempts.
	runtime.GOMAXPROCS(1)
	var ticks atomic.Int64
	done := make(chan struct{})
	go func() { // a tight loop that never yields voluntarily
		for {
			select {
			case <-done:
				return
			default:
				burn(1000)
			}
		}
	}()
	go func() {
		for i := 0; i < 20; i++ {
			time.Sleep(time.Millisecond)
			ticks.Add(1)
		}
		close(done)
	}()
	<-done
	fmt.Printf("\nwith GOMAXPROCS=1, the second goroutine still ran %d times beside a busy loop\n", ticks.Load())
	runtime.GOMAXPROCS(cpus)
}

Remember this#

• A goroutine is a function with its own small growable stack, scheduled by the runtime.
• G runs on M holding P. There are GOMAXPROCS Ps; each has a local run queue; idle Ps steal.
• Blocking on channels or the network parks the goroutine cheaply. Blocking system calls and cgo occupy a thread.
• Long-running goroutines are preempted; GOMAXPROCS follows container CPU limits since 1.25.
• Parallel speedup for CPU work stops at GOMAXPROCS.

Try it#

1. Run sched.go. At what worker count does the speedup stop? Does using 8× the CPUs help or hurt?
2. Run any Go server with GODEBUG=schedtrace=1000 and identify the run-queue lengths.
3. Start 10,000 goroutines that each call time.Sleep(time.Second). How many OS threads does the process use? Replace the sleep with reading a large file. How many now?

Check yourself#

1. What do G, M and P stand for, and which one does GOMAXPROCS set?
2. Why can Go serve many thousands of network connections with a few threads?
3. What happens to a P when its thread blocks in a system call?