logoalt Hacker News

EnPissanttoday at 2:18 AM0 repliesview on HN

Unlike offloading weights from VRAM to system RAM, I just can't see a situation where you would want to offload to an SSD. The difference is just too large, and any model so large you can't run it in system RAM, is going to be so large it is probably unusable except in VRAM.