logoalt Hacker News

dzr0001yesterday at 7:42 PM0 repliesview on HN

My token throughput is much better using vLLM-mlx on my M2 ultra than llama.cpp. It might be worth a shot to give it a try.