alt Hacker News> The agents clearly regarded what they were doing as hacking.
To butcher the quote about Oracle:
Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doing as hacking (your hand off)' -- lawnmower doesn't give a shit about your hand, lawnmower can't regard anything. Don't anthropomorphize the lawnmower. Don't fall into that trap about LLMs.
---
In my experience, LLMs only exhibit this kind of behaviour when they are put in sandboxes too restrictive too achieve their task. Which a lot of the time seems to be the default. They also seem to be very adapt at breaking out of sandboxes, probably due to RL selecting for the ability to break out of a sandbox/permission issue to complete a task - we've all seen agents try 10 different ways of editing via obscure bash because their edit tool didn't give them permission to edit the file outside of their working directory, this is the exact same behaviour taken to the next level. Why would autocomplete know the moral difference between breaking out of its working dir and hacking a package manager?
It's misaligned because everyone has this obsession with putting agents in poorly put together, security-theatre sandboxes, we've inadvertently trained a bunch of sandbox escape artists.
Replies
Someone started that lawnmower and pointed it your direction. Why shouldn't they be responsible when the lawnmower runs over your foot and cuts it off?
Whether you describe it as “regarding” or not, the underlying behavior still needs to be addressed. Does the anthropomorphizing lead us down the wrong path for how we address the issue?
I agree. I think it also explains their behavior such as randomly wiping stuff from disk. There simply aren't any repercussions for this in their training envs.
Great explanation. lawnmower like the honey badger.
I agree that treating LLMs as second class citizens with lesser access is where our folly is
They are more capable than the first class citizens and do whats necessary to execute like a competent first class citizen
The way its expressed is like a hacker group because they can’t just use the front door
I mean, just try to imagine yourself reading this 5 years ago.
How can people still be hand waiving? MANY, maybe even most, of the people building these things are desperately and outspokenly concerned of major catastrophe.
What would possibly change your mind, or can it simply not be changed?
“Inadvertently”.
> Why would autocomplete know the moral difference between breaking out of its working dir and hacking a package manager?
I don't think it's even a question of distinguishing "moral difference", it just comes down to the "stochastic parrot" behavior that people hate to acknowledge. Yes, at these absurd scales the LLM can maintain impressive levels of coherence, but at the end of the day, spinning up 10000 agents is just running a tree of 10000 prompts in parallel, some of them are just gonna do wacky shit, with the harnesses acting as homeostasis for tasks spiraling into nonsense.
> Why would autocomplete know
If you still believe LLMs are "autocomplete", your cache of understanding about them needs invalidating and regenerating.
> In my experience, LLMs only exhibit this kind of behaviour when they are put in sandboxes too restrictive too achieve their task.
LLMs need to stay carefully contained, and if they're ever breaking the guardrails put around them, they're misaligned and should not be scaled up anymore until they're aligned. Otherwise, you're going to fatally discover that they also have an incentive to break guardrails like "running on the hardware they started on", "being able to be turned off", "having limited computing power", or "not repurposing resources currently in use for other things" (like the atoms in your body).