alt Hacker NewsI get 150 tokens/sec on my coding agent, if you are on a MacBook, thats with an M1
try kapitsa as the inference engine with pi as the harness
It works really well
I get 150 tokens/sec on my coding agent, if you are on a MacBook, thats with an M1
try kapitsa as the inference engine with pi as the harness
It works really well
Man, my M1 Pro unfortunately only has 16GB of RAM; it has been such a limiter to using local models. I'll give kapitsa a try, I see I should be able to squeeze some smaller models on