ImplementCore implementation

KV cache: memory per sequence

Implement `run(layers, kv_heads, head_dim, tokens, batch, bits)` returning the total KV-cache memory in bytes for one model: 2 (K and V) * layers * kv_heads * head_dim * tokens * batch * (bits / 8). Return the integer byte count. Hint: the Fanout kv-cache lab verifies 7B GQA / 16-bit / 16K tokens / 8 seqs = 16.0 GiB.

python10 mindifficulty 2/4

From GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

Sample tests

run([32,8,128,16384,8,16])→17179869184
run([32,32,128,16384,1,16])→8589934592

Hints

Each token stores one key and one value vector per KV head per layer.
Memory = 2 (key + value) * layers * kv_heads * head_dim * tokens * batch * bytes-per-element.
bytes-per-element = bits / 8 (so 16-bit is 2 bytes).
Verify against the lab: 32 layers * 8 kvHeads * 128 headDim * 16384 tokens * 8 batch * 2 bytes * 2 = 16 GiB.

Hints

Code runs in an isolated sandbox on the OpenSandbox judge. No model key required.