I don't think it's that difficult. Their servers are stateless too. S3 is easy to migrate.

Database is more difficult, but tons of people have done it successfully.... meanwhile people who host their own LLMs are relatively small in number in comparison.

Most companies don't do their own data centers mainly because it is more expensive and less reliable. It's something they can just pay for the problem to go away. The calculus for hosting your own LLM is probably similar.

Even Stripe who built their own coding agents and has tons of money/resources still decides not to host their own LLMs.

Still, many people will prefer open-weight models. It is similar to how we prefer linux but still use AWS/Render/and whatever. It doesn't lock us in, and we can move providers if we want to.