Skip to main content
All field notes

Inference

The KV cache formula: LLM inference memory

Deriving the memory cost of attention state and why it dominates long-context serving.

Study-clone note: the original long-form article is intentionally not mirrored here. This page reproduces the reading layout with a short original briefing on the same topic.

Why this topic matters

Engineers who work through inference material in a structured way build judgment that ad-hoc skimming rarely produces. The Fanout approach sequences the topic so each piece of context is in place before the next one arrives.

The short version

Deriving the memory cost of attention state and why it dominates long-context serving. Ordered study — with the related Daily papers, labs, and roadmap nodes — turns a topic like this from a pile of links into a path.

Where to go next