PidokuInfra
14Research and Frontier
On this page

Research and Frontier

ExpertModule 1411 topics~11h 15m

Topics, in order

01 How to Read an Inference PaperThe transferable skill of this section. Learn this and files 02-08 become a reading list rather than a syllabus. Advanced 1h 15m 02 Attention ResearchSection VII.10 covers what you need. This line of work is done from a practitioner's perspective; the remaining research is kernel engineering for new hardware. Advanced 1h 03 KV Cache ResearchThree of seven are solved, two are established, and the interesting research is concentrated in the one with the worst failure mode. Advanced 1h 04 Quantization ResearchReading a paper without knowing which era it's from wastes time. An Era 1 paper proposing a weight quantization method is historically interesting; an Era 3 paper about FP4 training is … Advanced 1h 05 Decoding ResearchThese are usually studied separately but interact: inference-time scaling makes decode speed matter much more. Advanced 1h 06 MoE ResearchNode-limited routing is the most instructive item here. It's an architecture constraint introduced specifically to make serving efficient — an example of inference requirements influencing … Advanced 45 min 07 Long-Context ResearchMost research addresses 1 or 2. Problem 3 is the one that determines whether the other two were worth solving, and it's the least studied. Advanced 1h 08 Scheduling and Systems ResearchThis is the pattern from Section XIV.01: exact techniques with no downside adopt fast. Systems work is mostly exact. Advanced 1h 09 NVIDIA Architecture TrendsThe H100→H200 transition is instructive: no new compute, 43% more bandwidth and 76% more memory, and it was a large inference improvement. That's evidence that the industry recognized … Advanced 1h 10 Alternative AcceleratorsThe honest position: NVIDIA is the default for good reasons, and the alternatives are viable for specific situations. This file is about identifying those situations. Advanced 1h 15m 11 Memory-Centric and Disaggregated FuturesThe final file. It looks at where the field is heading, and closes the loop on the idea this curriculum opened with. Advanced 1h

About this module

Goal: be able to read the field, judge what matters, and know where it’s heading.

This section is deliberately different from the others. It teaches a skill — reading and evaluating inference research — and then applies it to survey the active areas. The specific papers will date; the skill won’t.

Files#

#FileLevelTime
01How to read an inference paper ★Advanced75 min
02Attention researchAdvanced60 min
03KV cache researchAdvanced60 min
04Quantization researchAdvanced60 min
05Decoding researchAdvanced60 min
06MoE researchAdvanced45 min
07Long-context researchAdvanced60 min
08Scheduling and systems researchAdvanced60 min
09NVIDIA architecture trendsAdvanced60 min
10Alternative acceleratorsAdvanced75 min
11Memory-centric and disaggregated futuresAdvanced60 min

The honest framing#

flowchart LR
  P["1. Paper published"] --> V["2. Independent validation<br/>6-18 months"]
  V --> E["3. Lands in a major engine"]
  E --> F["4. Becomes a config flag"]
  F --> D["5. Becomes the default"]
  P -.->|"most papers stop here"| X["Never changes production"]

  class P neutral
  class V queue
  class E io
  class F compute
  class D memory
  class X warn
Most inference papers do not change production practice.

Of those that do, the pattern is usually:
  1. paper published
  2. 6-18 months of independent validation and engineering
  3. implementation in a major engine
  4. becomes a config flag
  5. becomes the default

PagedAttention took ~12 months from paper to ubiquity.
FlashAttention took ~18 months.
Speculative decoding took ~24 months and is still not universal.
KV eviction methods are 2+ years in and still not production-standard.

→ your job is to identify which papers are on that path, and when
  to pay attention.

File 01 teaches how to make that judgment. Files 02-08 survey the active areas with that lens. Files 09-11 look at where the hardware is going, which determines what will matter next.

How to use this section#

Read file 01 carefully. It’s the transferable skill.

Skim files 02-08 to know what exists. Return to one when you hit its specific problem.

Read files 09-11 once a year. The hardware roadmap determines which software techniques become important.

esc

↑↓ navigate↵ openesc close