alt Hacker NewsOh, that's a tough one. Do you have a fast drive? If you do you might try a 3 to 5B active MOE model, use the rest as cache to stream weights from NVMe. That would be your best bet. You'll be in the 'optimization league' ;)
Oh, that's a tough one. Do you have a fast drive? If you do you might try a 3 to 5B active MOE model, use the rest as cache to stream weights from NVMe. That would be your best bet. You'll be in the 'optimization league' ;)