alt Hacker NewsIf you don't use any LLM router, it's infinity times faster, and infitesimally as targetted.
Replies
toxik • today at 6:01 PM
Yeah, this DIY MITM attack is entirely avoidable.
If you don't use any LLM router, it's infinity times faster, and infitesimally as targetted.
Yeah, this DIY MITM attack is entirely avoidable.
I have GoModel running at home, being deployed at work. There are many good reasons to run a gateway, some include
- being resilient to upstream provider downtime
- centrally managed config/controls
- visibility and allocation into usage and costs
We are deploying a gateway at work primarily for these three.
At home, GoModel's virtual models have been super helpful for testing out new open weights. I can change the model deployed in vLLM without having to change any of my downstream tools, which get a stable name.
https://gomodel.enterpilot.io/docs/features/virtual-models