It seems like there is no concept of deployment, or even A/B test, what works on presumably claude employee's laptop for the hour they spent testing it will ship immediately to everyone.

I mean, yes, even testing in production with some of your customer is better than.. testing with ALL of your customers?