Skip to main content
fanout

Calculations run on this device. Scenario values are not sent anywhere — the engine is fully local.

inference calculator · free

Prefill–Decode Roofline

Plot the two inference phases against a roofline and read off the regime.

Are you compute-bound or bandwidth-bound at this context length?

Engine 1.0.0 · calculation
102000
3.35
2
1
Ridge point
93.1 FLOP/byte
Bound
memory
Achievable TFLOP/s
6.7
Phase
decode

Decode is usually memory-bound — KV/bandwidth dominates.

AI 2 vs ridge 93.1

Method

  • Deterministic local model of the mechanism — knobs recompute outputs instantly with no network.
  • Numbers are teaching approximations, not production capacity guarantees.

Sources

Keep going in inference

inference calculatorKV Cache CalculatorWhy do long context and batch size compete for the same GPU?inference calculatorGPU Model FitDoes this checkpoint actually fit on that GPU?inference calculatorLatency Numbers Every Engineer Should KnowWhich operation dominates this request's budget?