alt Hacker NewsHow does this choose which models to use with any arbitrary set of model providers to work from? And why is an openrouter necessary for self-hosting?
How does this choose which models to use with any arbitrary set of model providers to work from? And why is an openrouter necessary for self-hosting?
To the point of using buckets of models: as long as there's >0 models available in a bucket, and we can order models in order of fit, we're resilient to different sets of models being available. With that said obviously cutting out some models has a much larger effect than others.
OpenRouter isn't strictly necessary but it does make it easier to not have to set up accounts/API keys with several different providers to get started.