just use an api gateway (ie., a reverse proxy for apis), eg., envoy. This should also be connected to observability and finops style management anyway rather than leaving it to model providers.