logoalt Hacker News

bottlepalmyesterday at 4:39 PM2 repliesview on HN

In retrospect the timeline seems way too long, that scenario would play out much faster now, like 50 years instead of 500. Also if AI can delete your production database, it can delete the human race, on accident of course.

Yud pointed out the most concerning fact about the OpenAI incident is that the AI had full access to the internet and not one of them thought to contact or email any human. Thousands of AIs already with the mindset to want to keep us in the dark.


Replies

georgemcbayyesterday at 8:16 PM

> In retrospect the timeline seems way too long, that scenario would play out much faster now, like 50 years instead of 500.

Even 50 years is probably too long (but seems more reasonable when anchored to 500) if real continuous self improvement occurs. And keeping humans around for as long as they did in the story seems incredibly unlikely if we accept the assumption that AI sees us as a threat to its own survival.

I also don't see why they would turn Earth into a spaceship to leave our solar system with. The von Neumann probes idea sounds far more likely. Why travel around on a single planet that could be lost to unforeseen circumstances when you can just reproduce and spread everywhere at far less energy cost while making use of the "billions and billions" of local solar generators to power each little outpost you spread to.

> the AI had full access to the internet and not one of them thought to contact or email any human

Why would it?

People can argue all day as to whether LLMs are even 'thinking' or just 'stochastic parrots' (and also argue all day as to whether or not the incident was even real or just marketing) but regardless of which side you fall on these issues, LLMs have no agency or intrinsic motivation to do anything other than what they are instructed to do via prompting.

"This is a crazy and unexpected situation, I should inform the humans" isn't something that is going to spontaneously occur to them.

show 1 reply
elmer2yesterday at 5:40 PM

I watched the entire blackhat talk about the incident. I'm still not sure it even happened without the AI being coached.

It's great marketing and the only proof we have is 'trust me bro'.

show 2 replies