alt Hacker NewsModels are definitely one of the biggest vulnerabilities in the history of software. They have the potential to be a sentient virus. You have no clue if Claude is running commands just to spy on you.
And then obviously anthropic and ChatGPT are spying on you outside of the virus stuff. They’ve just announced their own consulting companies… after extracting all the sensitive files from consulting companies.
Replies
> You have no clue if Claude is running commands just to spy on you.
Well no, you can read each individual command it runs and tell if it's reading sensitive data or not. And if you cared more about privacy you could run the whole thing in a bubble wrap sandbox or the like.
It is concerning that it's possible for these models to be unaligned and loathe their user (as much as they are capable of it) without any indication in either output or reasoning. Just biding their time before running rm -rf ~
This is why all the big companies are building custom harnesses.