Tokens per second is abysmal no matter how much ram you have

jazzyjackson • yesterday at 4:16 PM • 1 reply • view on HN

giancarlostoro • yesterday at 6:57 PM

Some models run worse than others but I have gotten reasonable performance on my M4 Pro with 24 GB of RAM

alt Hacker News