alt Hacker NewsI don't understand why so many comments here are so confident that this is all marketing, that rogue is just hype, that agents are just simple tools, etc. If a bunch of nuclear engineers were going to the news and saying "Our reactor is dangerously close to a meltdown - we need government intervention now!" would your response be that they're just hyping up boring old power generation technology?
Replies
These companies are building software. That doesn't work very well. The output it produces does make sense at times, but there are times when it doesn't. And instead of fixing that, or admitting it can't fixed, they started bolting actuators to them, executing actions online (for now) based on the output of their buggy software.
And when this results in actuators executing some bad actions they scream in horror "AI went rogue! It escaped the containment!!! It's going to kill us all!!!"
Go fix your software before you let it do stuff online or IRL. It's not "Terminator", it's just bad QC.
I’d roll my eyes if the engineers stated that they didn’t design the reactor to melt down, and that it simply developed rogue meltdown-desiring behavior on its own, and I would also wonder about negligence if they claimed that nobody could have anticipated this (given that, like with botnets and viruses, we have decades of knowledge and experience regarding reactor meltdowns)
Go and get one of their models to hack something, it won't do it, why?
They have claimed this happened during a "training run", but why are they training on systems connected to the internet?
That's why people are skeptical.
You wouldn't be suspicious if the nuclear engineers kept on feeding their reactor and building more powerful reactors while they went crying to the news?