> It is very frustrating.
Not just frustrating, it can also be dangerous. What if you trust Claude with certain security-sensitive things (because experience), and suddenly you cannot trust it anymore like that?
It’s happened before and it will happen again. At least now you know.
Then you should have evals.
Maybe don't trust something with non-deterministic output for critical level security?
> What if you trust Claude with certain security-sensitive things
then $corp should remove your access to these security-sensitive things ;)
It literally could have been rolled out in the middle of that brain surgery experiment on a mouse. Woops.
It seems to me the answer is in your question.
You shouldnt do that in the first place