alt Hacker NewsTwo days sounds like a lot, both llama.cpp and openvino only takes a few minutes to compile on any decent modern cpu.
Replies
It also even easier to get working and integrate into your system in a sustainable manner with NixOS. Do it by hand or throw an LLM at it, it will get you a declarative patch for your NixOS config that brings llama-cpp into your config that you can review and add under version control (no random `make install` build artifacts contaminating your system, no wondering "what was it that I ran? what are all these files? how do I do the same with a newer version?" a couple months later). There's also likely some build cache where Nixoids have already build what you want.
I had a great experience with llama-cpp with Nvidia backend on NixOS.
(Sorry for being that guy.)
the full set of llama.cpp binaries builds in under 5 minutes with an unmodified build workflow straight from their github page on a literally ten year old dual xeon.