logoalt Hacker News

avadodinyesterday at 11:23 PM0 repliesview on HN

You're arguing for a very specific range of weights but many slightly smaller and slightly larger models have been released including QAT and MoE versions.

An old nVidia brand card with 8GB is more than enough to see those models running at usable speeds and accuracy.