Ever heard of social engineering? Also, models nowadays are way sharper than they were even a year ago. They’re not going to make stupid mistakes like that unless you basically ask them to. GPT-5.x for example would bend over backwards to avoid even reading your passwords into context.
> Ever heard of social engineering?
Oh wait, I thought these things were super smart. I didn't expect "social engineering" to work on them.
> models nowadays are way sharper than they were even a year ago.
You are missing the point. If the thing can solve complex math problems and at the same time be so dumb as to fall for "social engineering", then that means that it is not "smartness" or "reasoning" that is helping it to solve those problems. Just some form of advanced, but yet dumb, search algorithm.