07Data Center and Frontier

Data Center and Frontier

ExpertModule 074 topics~3h

Topics, in order

01 GPU GenerationsNVIDIA now releases a new data-center GPU generation roughly every year. Each one moves the same handful of levers: more memory capacity, more memory bandwidth, a smaller number format on … Advanced 45 min 02 Other AcceleratorsNVIDIA GPUs are the default for AI, but not the only option. Alternatives fall into a few families: other GPUs (AMD, Intel), cloud vendors' own chips (Google TPU, AWS, Microsoft, Meta), … Advanced 50 min 03 Racks, Power and CostAt scale, the GPU chip is not the hard part. Electricity, cooling and networking are. A modern AI rack draws as much power as a small neighbourhood, must be liquid-cooled, and is worthless … Advanced 50 min 04 Where This Is GoingSpecific products will be obsolete in a year. The pressures that shape them will not. Six trends follow directly from physics and economics you have already learned: memory is the … Advanced 35 min

About this module

The last module zooms out: from one chip to product generations, to competing designs, to the buildings full of them, and finally to where the technology is heading.

#	Lesson	The question it answers
01	GPU Generations	What actually changes from one generation to the next?
02	Other Accelerators	What are the alternatives to NVIDIA GPUs, and when do they win?
03	Racks, Power and Cost	What does it take to run thousands of GPUs, and what does a GPU-hour cost?
04	Where This Is Going	Which trends are physics, and which are fashion?

Specifications here age quickly. Each lesson separates the durable reasoning from the numbers that will be out of date by the time you read them.