Simulation run on this device. Scenario values are not sent anywhere — the engine is fully local.
inference simulation · free
Continuous Batching Dispatcher
Schedule sequences into decode slots and watch padding waste fall away.
Why does continuous batching lift GPU utilization?
Engine 1.0.0 · simulation
1128
1200
5100
164
- Continuous util (toy)
- 13%
- Static util
- 25%
- Slots
- 32
- Iter time
- 25 ms
Continuous batching fills holes as sequences finish — higher GPU util than static batches.
slots 32, arrival 20/s
Method
- Deterministic local model of the mechanism — knobs recompute outputs instantly with no network.
- Numbers are teaching approximations, not production capacity guarantees.