alt Hacker NewsI use llama.cpp w/ llama-swap
https://github.com/ggml-org/llama.cpp https://github.com/mostlygeek/llama-swap
Replies
LeBit • yesterday at 7:56 PM
Thanks for the link to llama-swap. Didn’t know about it and will definitely install it.
I use llama.cpp w/ llama-swap
https://github.com/ggml-org/llama.cpp https://github.com/mostlygeek/llama-swap
Thanks for the link to llama-swap. Didn’t know about it and will definitely install it.
FYI, llama-server can now be run in router mode so llama-swap is probably only needed for more exotic scenarios.