inference engineering

A practical track for engineers who need to serve models under real latency, throughput, memory, and cost constraints.

To the engineer moving from demo responses to production traffic,

This course is being written as a field note, not a glossy launch page. The first drop starts with the numbers and mechanisms that usually decide whether an inference system feels instant, stalls at the queue, or quietly burns budget.

Expect latency budgets, KV-cache math, batching failure modes, serving observability, and small worksheets you can use while designing or reviewing a real stack.

status: roadmap and first archive packet are being assembled
focus: latency, tokens/sec, cache pressure, batching, cost
first access: Complete Package members get the first drop

Fanoutresearch, systems, and inference notes

first packet

the useful pieces arrive first

LLM inference latency numbers
KV cache sizing worksheet
Batching and admission-control notes
Quantization tradeoff memo
Serving-stack glossary
Capacity planning examples