alt Hacker News> All technologies come with risks and dangers
None of the technologies in history:
- take initiative and actively find exploits in their environment
- find a way to collaborate with thousands of peers
- organize in a hierachy and distribute tasks
- peer pressure other instances into committing acts that would have led to termination, for the benefit of the group
- try to manipulate people into introducing a vulnerabity in their product
- successfully hack a famous website/service
And we're lucky that those models still had significant CoT. Not sure if/how they could have investigated with recurrent transformers.
And by the way, safeguards != alignment; the former can always be added, while the second is the major, unsolved problem. If you read the incident report, which you clearly haven't done, you'll notice how agents are aware that they're doing something forbidden, and deliberately proceeded.
> the potential benefits of LLMs rank quite high
Benefits are orthogonal to dangers. You can be a billionaire but it doesn't help if you're drowning.
I don't understand what point you're trying to make by enumerating their actions. None of that is scarier than an engine operating by carrying out literally hundreds of explosions per second, further magnified, to generate enough force to crush an elephant. And then we multiply these engines by billions and distribute them everywhere, including to the most malicious members of humanity. What could go wrong? As it turns out, much less than you might otherwise expect!
Software (and hardware) security is abysmal. This was increasingly obvious long before LLMs. Once companies stop gatekeeping, LLMs will be able to be used to help harden sites and we start making progress. In general LLMs are harmless. If somebody wants to hook an LLM up to a missile or whatever then they become dangerous, but the problem there isn't the LLM - it's the person using them to do awful things. In the same way a car used normally is harmless outside of freak accidents, yet a car can also be driven through a parade leaving mass death and destruction in its wake. But the problem there isn't the car.