LLMs are inherently unverifiable and untrustworthy. The training data may be incorrect, malicious, censored, or modified to serve the parent company. The model itself is a black box. Safe input is impossible: there is no way to escape natural language or separate command and data into separate streams. The output is stochastic, better on average than any algorithm could ever be, but with no guarantees on individual cases.

All of that is fine, because an LLM is a text-only interface. It cannot harm the computer because it cannot perform actions.

Why the fuck would you give it a shell?

Obviously, it's to have a product that can do anything as quickly as possible. You can make a shell-based harness in a day. Since the competition has a shell-based harness, every AI company that wants to keep up has to as well. They're stuck forever trying to plug all the holes in an attack surface as broad as written word. Solving this impossible problem requires ideas as brilliant as using a second untrustworthy LLM to validate the output of the first untrustworthy LLM that is following instructions from the internet.