> Banks, Biglaw, and the Pentagon all do it in the cloud.

In _a_ cloud: their own virtual private cloud. They also have enough power to negotiate contracts with strong privacy provisions.

I like to buy specific brand of soap. I don't want them to know that, it is my right and so is running local LLM "wasting" money on local inference to keep track of my stack of soap.

Those companies also have data sharing/use agreements that they can get from Cloud AI providers due to their size and spend. The secrecy and data protection is largely what they are paying for. Those types of agreements just aren’t available to individual customers. It’s only when you’re spending $$$ that it becomes worth it for the provider.