03The Programming Model

The Programming Model

IntermediateModule 035 topics~4h 50m

Topics, in order

01 Kernels, Threads, Blocks, GridsYou give a GPU a kernel — a function — and ask for it to be run by a huge number of threads. Every thread runs the same code; the only difference between them is an index that says "which … Intermediate 1h 02 Your First Kernel, Called from GoA real GPU program has two halves. The kernel is written in CUDA C and compiled for the GPU. The host program — ours is in Go — allocates GPU memory, copies data in, launches the kernel, and … Intermediate 1h 30m 03 Host and Device MemoryThere are two memories — the host's and the device's — and a pointer into one is meaningless in the other. You must allocate on the right side and copy between them explicitly. The four … Intermediate 50 min 04 Asynchronous Execution and StreamsLaunching a kernel does not run it. It puts a command in a queue and returns immediately; the GPU works through the queue on its own time. The host only waits when it synchronizes — … Intermediate 50 min 05 The Software StackBetween your program and the GPU's transistors sit several layers: frameworks, libraries of ready-made kernels, the CUDA runtime, the driver, and — in containers — a small piece of glue that … Intermediate 40 min

About this module

You know what the hardware is. Now: how do you tell it what to do?

This module teaches the CUDA programming model — the one nearly every other GPU API copies — and shows how a Go program drives it.

#	Lesson	The question it answers
01	Kernels, Threads, Blocks, Grids	How is work described to a GPU?
02	Your First Kernel, Called from Go	What does a complete GPU program look like?
03	Host and Device Memory	Where does data live, and how does it move?
04	Asynchronous Execution and Streams	Why does a GPU call return before the work is done?
05	The Software Stack	What sits between `go run` and the silicon?

Lesson 02 needs an NVIDIA GPU and the CUDA toolkit to run; everything else runs anywhere. If you have no GPU, read 02 carefully and do its simulator exercise instead — the concepts carry.