alt Hacker NewsUnfortunately, the MI300X is an OAM module. The MI350P is the one you want: It's a PCIe card, but it has less memory: 144GB.
Luckily, DeepSeek V4 Flash will run in 144GB too because it's 256 MoE exports are native MXFP4 quantized.
Replies
Just want to add that while the MI350P is a PCIe card, it is designed for servers. It just has a heatsink (with no fan) which the powerful full-case fans of a rackmount server are supposed to cool. So while the MI350P is certainly more attainable for us regular folk due to its formfactor, we won't be able to just drop it into our gaming PCs like a regular graphics card.
That being said, if you're dropping tens of thousands of dollars on graphics cards then picking up a rackmount case to go around the card is pretty insignificant.
How do you figure that?
When they just loaded the weights alone, it was taking 156GB in vLLM. After warm-up and adding a KV cache pool, it took over 200GB.
And this implementation is already cutting down the 1M token context window you would normally get.