I would figure that in an enterprise setting (where the stakes are highest) having an AI gateway sit in the middle of the prompt to the LLM and its response would mitigate a lot of the concerns over using untrusted models (all of them).

This is one of the insane things about the popularity of LLMs that continues to astound me. It is SO EASY for an LLM provider to collect vast amounts of PII or misdirect their customers in extremely subtle ways without noticing. It's already known that most LLM users don't fully read its (verbose by default, possibly on purpose) output and it's also known that LLM providers train on chats.