I’m using llama-swap because it can manage arbitrary backends, not just llama-server instances. I have llama.cpp chat and embedding models running alongside whisper-server all behind a single endpoint with per-model TTLs so they don't fight over the limited vram I have available on this box. Native routing could replace the llama.cpp part but not whisper so I guess I'm exotic ;)

I'm running it in router mode, but people on Reddit were recommending people use llama-swap instead. Am I missing something by using router mode?