Mostly yes, you are wrong.

Active parameter count really drives inference speed not ram requirements.

For this model, assuming you want the full 1mm context length at 4-bit quants, you need something like 175 GB total VRAM.

~150GB of that will be weights (varies a bit by quant format) then ~3-5GB for kv cache (might be lower here, I think they're using mostly HCA compressed cache) and then I'll allocate a couple gb for overhead/concurrency (say another ~10GB as a safety buffer).

---

So the good news is that it comfortably fits into 2x128 at 4-bit, and we can realistically stretch to a 6-bit quant and make it just fit (~244GB needed for the full 1mm context).