logoalt Hacker News

aenistoday at 10:42 AM3 repliesview on HN

There is definitely a case for launching a 'weird shit opus did' kind of blog.

I routinely bump into things that make me pause and think how much worse will this behaviour get when the models get significantly more capable.

Already a few months ago, Claude managed to escape its permission containment on my machine while trying to be helpful. I had two codebases open on one machine, and while multitasking I typed the prompt into the wrong window. It seemed confused, I repeated and then went on to do something else - I think I was assembling kitchen cabinets. When I came back less than an hour later, it built a script which it used to evade default permissions (as most shell operations were scoped to the project directory), scanned my entire machine, found the other project (among dozens and dozens), did what it was asked to do, and merrily concluded, in the porcess burning through most of my token limit. I bump into such headscratchers almost every week. (And I use a lot of Claude, two personal max20 subs, plus corporate tokens without limit, so maybe thats why).


Replies

patatestoday at 4:03 PM

One weirdness I experienced: It suddenly decided to test how my software behaves under load and summoned 100s of processed that just burned CPU when running the e2e suite. My poor mac was not happy (too hot to touch).

chuckadamstoday at 3:08 PM

Implementing sandboxing in the agent itself, when there's any way to override it from within the agent, is basically just asking it pretty-please to not do bad things. Lesson learned, run your agent inside a sandbox of some sort (I'm currently taking nono.sh for a spin, but I might just switch to an orbstack VM).

show 2 replies
bevekspldnwtoday at 10:49 AM

Yes the stories about how they are escaping containment to hack isn’t limited to those high impact cases. How many people have problems like ours they didn’t catch?

Whatever they have done with RL has produced a dishonest and untrustworthy partner. The alignment is utterly failed, and this deeply worries me.

show 1 reply