alt Hacker NewsMan, my M1 Pro unfortunately only has 16GB of RAM; it has been such a limiter to using local models. I'll give kapitsa a try, I see I should be able to squeeze some smaller models on
Man, my M1 Pro unfortunately only has 16GB of RAM; it has been such a limiter to using local models. I'll give kapitsa a try, I see I should be able to squeeze some smaller models on
kapitsa is customized well to just a small selection of models that are in vogue right now, focusing on qwen and nemotron
compared to llama.cpp that tries to cater to all but does so only averagely
check the catalogue and see if one of the quantizations fit