alt Hacker NewsYeah the only way to run 4 sticks of DDR5 decently is with Intel. It's a bit of a shame that you can't cram enough RAM to run big models.
The most I could get running on 10GB VRAM + 96GB RAM was a REAP'd + quantized version of MiniMax-M2.5
Replies
I’m running 128gb on a 9550x now with 4x32gb sticks and it’s terrible. It’s unstsable, post time is about 2 minutes (not exaggerating)and I’m stuck at a lower speed. I’m considering just taking 2 of the sticks out and working with 64gb and increasing my swap partition. The nvme drive is fast at least.
This is my first time off intel and I have to say I don’t understand the hype.
I had the same issue with Intel. It's not guaranteed there either.
Threadripper is a good alternative. No point having a lot of dual channel ram for LLMs, too slow
Got it running with 4800MT/s and literally 30 minute boot times in an AM5 machine. The 30 minute boot time could be worked around by enabling the (off-by-default) memory context restore option in BIOS, but it really made me think something was broken and it wasn't until I found other people talking about 30 minute boot times that I stopped debugging and just let it sit for an eternity.
It's so bad. I don't get why they sell AM5 motherboards with 4 RAM slots.
At least that system has been running well for like two years. But had I known that the situation is so much more dire than with DDR4, I would've just gotten the same amount of RAM in two sticks rather than four.