Calculations run on this device. Scenario values are not sent anywhere — the engine is fully local.
inference calculator · free
KV Cache Calculator
Watch context length turn into GPU memory: layers, KV heads, precision, tokens, and batch, live.
Why do long context and batch size compete for the same GPU?
Engine 1.0.0 · calculation
16,384 tokens
Cached tokens per active sequence.
8
The batch multiplier applied to the cache.
16.0GiB
64.0 GiB remains for weights, activations, and runtime overhead.
- Total KV cache
- 16.00 GiB
- Per token / sequence
- 128 KiB
- GQA ratio
- 4:1
- Sequences in 80 GiB
- 40
2 × layers × KV heads × head dimension × tokens × batch × bytes
Method
- Counts one key and one value per layer, per KV head, per token, per active sequence, multiplied by element bytes.
- GQA reduces KV heads relative to query heads; the readout shows how sharply that cuts memory.
- Deterministic arithmetic only — a model of the mechanism, not a benchmark promise.