If you need a GUI, Unsloth Studio and LM Studio are both great frontends for llama.cpp. If you don't need a GUI, llama.cpp is the business for single-user deployments. Easy to use, always gets new model support very quickly, built-in Hugging Face client/cache support, works on probably everything (Mac, ROCm, CUDA, Vulkan, etc.).

I use llama.cpp w/ llama-swap

https://github.com/ggml-org/llama.cpp https://github.com/mostlygeek/llama-swap

try oMLX or vMLX - both great projects that offer some amazing performance optimizations for Apple Silicon that utilize UMA and NVME caching efficiently.

https://omlx.ai https://vmlx.net

That said, it's been a few weeks since I've looked so maybe llama.cpp has those features now... they really do move that quickly.

Jan is so much better.

The link in that discussion has a section about that

lm studio