Llama-server which is part of llamacpp does this for a few months now