Congrats for shipping!
Have you seen LiteLLM Gateway before? As a bonus it supports a lot of different providers.
https://docs.litellm.ai/docs/simple_proxy
Yes. Because my original purpose is to use third party LLMs in JetBrains AI Assistant, it doesn't provide Ollama or LM Studio endpoints as required. So I decide to develop one on myself.
alt Hacker News
Yes. Because my original purpose is to use third party LLMs in JetBrains AI Assistant, it doesn't provide Ollama or LM Studio endpoints as required. So I decide to develop one on myself.