04Running It

Running It

IntermediateModule 045 topics~4h 10m

Topics, in order

01 The Telemetry PipelineBetween your process and a graph there is a pipeline with four stages: collect (scrape or receive), process (enrich, filter, sample), store (a database shaped for that signal), and query (a … Intermediate 55 min 02 DashboardsA dashboard is an answer to a question, laid out so that a tired person can read it in ten seconds. Good dashboards are layered: one overview that says whether users are happy, then one … Intermediate 40 min 03 Alerting on SLOsAn alert should mean: a human must act now, and here is what users are experiencing. That rules out most alerts people write. "CPU above 80%" is a cause that may or may not hurt anyone. "1% … Intermediate 55 min 04 Sampling and CostTelemetry is a cost centre that grows faster than traffic unless you manage it. Each signal has one cost driver: metrics cost per active series, logs per gigabyte, traces per span. So each … Intermediate 50 min 05 Debugging an IncidentDebugging with telemetry is a search. Start from the symptom the alert reported, and narrow by repeatedly asking one question: what is different about the bad requests compared with the good … Intermediate 50 min

About this module

Emitting telemetry is the easy half. This module is about the system that receives it, the screens and alerts built on it, what it costs, and how to use it when something is on fire.

#	Lesson	The question it answers
01	The Telemetry Pipeline	Where does telemetry go after it leaves the process?
02	Dashboards	What belongs on a screen, and in what order?
03	Alerting on SLOs	When should a human be woken up?
04	Sampling and Cost	How do I keep the useful 5% and afford it?
05	Debugging an Incident	How do I get from a page to a cause?

When you finish you can draw the pipeline for a real system, build a dashboard someone else can read under pressure, write a burn-rate alert, and budget the telemetry bill.