Skip to main content
fanout

Calculations run on this device. Scenario values are not sent anywhere — the engine is fully local.

inference calculator · free

Prompt Cache Savings Planner

Model shared system prompts as cached tokens and total the savings.

How much do reused prefixes shave off the bill?

Engine 1.0.0 · calculation
0100000
108000
0100
3
Cost no cache
$0.01350
Cost w/ cache
$0.00510
Savings
$0.00840
Hit rate
70%

Long stable system prompts amortize best with prefix caching.

prefix 4000 suffix 500

Method

  • Deterministic local model of the mechanism — knobs recompute outputs instantly with no network.
  • Numbers are teaching approximations, not production capacity guarantees.

Sources

Keep going in inference

inference calculatorKV Cache CalculatorWhy do long context and batch size compete for the same GPU?inference calculatorGPU Model FitDoes this checkpoint actually fit on that GPU?inference calculatorLatency Numbers Every Engineer Should KnowWhich operation dominates this request's budget?